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ABSTRACT 


The  need  to  deliver  high  quality  goods  and  services  in  less  time  and  at  a  lower 
total  cost  has  resulted  in  a  Revolution  in  Business  Affairs  throughout  the  Federal 
acquisition  community.  When  the  lowest  price  is  not  the  driving  factor  during 
procurements,  the  Federal  Acquisition  Regulation  allows  procurement  officials  to  make 
trade-offs  between  cost  or  price,  past  performance,  and  technical  merit  to  ensure  the  best 
value  to  the  Government.  One  of  the  tools  that  Government  procurement  officials  have 
for  reducing  procurement  risk  is  the  ability  to  request,  via  the  request  for  proposal, 
information  regarding  a  supplier’s  past  performance  and  to  use  this  information  in 
selecting  sources  of  supply. 

This  thesis  investigates  the  use  of  past  performance  information  in  the  Federal 
acquisition  process.  It  reviews  the  current  policies  and  best  practices  for  the  collection 
and  evaluation  of  past  performance  information.  It  then  categorizes  and  analyzes  the  past 
performance  protest  decisions  handed  down  by  the  Comptroller  General  from  October  1 , 
1997,  to  June  30,  2000.  Following  the  review  and  analysis,  the  overall  best  practices  and 
case  principles  are  linked  with  the  phases  of  the  Federal  acquisition  process  to  develop  a 
managerial  model  that  will  aid  contracting  officials  in  more  effectively  incorporating  the 
use  of  past  performance  information  into  their  acquisition  and  contracting  processes. 
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I.  INTRODUCTION 


A.  PURPOSE 

This  thesis  investigates  the  use  of  past  performance  information  in  the  Federal 
acquisition  process.  It  reviews  the  current  policies  and  best  practices  for  the  collection 
and  evaluation  of  past  performance  information.  It  then  categorizes  and  analyzes  the  past 
performance  protest  decisions  handed  down  by  the  Comptroller  General  from  October  1, 
1997,  to  June  30,  2000.  Following  the  review  and  analysis,  the  overall  best  practices  and 
case  principles  are  linked  with  the  phases  of  the  Federal  acquisition  process  to  develop  a 
managerial  model  that  will  aid  contracting  officials  in  more  effectively  incorporating  the 
use  of  past  performance  information  into  their  acquisition  and  contracting  processes. 

B.  BACKGROUND 

The  need  to  deliver  high  quality  goods  and  services  in  less  time  and  at  a  lower 
total  cost  has  resulted  in  a  Revolution  in  Business  Affairs  throughout  the  Federal 
acquisition  community.  The  thrust  of  this  movement  is  that  Government  procurement 
actions  should  be  modeled  more  along  the  lines  of  corporate  America.  As  corporations 
focus  more  on  their  core  functions,  they  are  becoming  increasingly  dependent  upon 
stables  of  reliable  and  preferred  suppliers  as  a  way  to  reduce  procurement  risk.  However, 
because  of  Governmental  concerns  for  competition  and  socio-economic  programs, 
Federal  procurement  officials  are  often  unable  to  follow  the  corporate  world’s  lead  with 
respect  to  preferred  suppliers. 
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One  of  the  tools  that  Government  procurement  officials  have  for  reducing 
procurement  risk  is  the  ability  to  request,  via  the  request  for  proposal  (RFP),  information 
regarding  a  supplier’s  past  performance  and  to  use  this  information  in  determining  and 
selecting  sources  of  supply.  When  the  lowest  price  is  not  the  driving  factor  during 
procurements,  the  Federal  Acquisition  Regulation  (FAR)  allows  procurement  officials  to 
make  best  value  determinations:  trade-offs  between  cost  or  price,  past  performance,  and 
technical  merit  to  ensure  the  best  value  to  the  Government. 

Best  value  in  negotiated  acquisitions  may  be  obtained  by  using  any  one  or  a 
combination  of  source  selection  approaches.  In  different  types  of  acquisitions,  the 
relative  importance  of  cost  or  price  may  vary.  For  example,  in  acquisitions  where  the 
requirement  is  clearly  definable  and  the  risk  of  unsuccessful  contract  performance  is 
minimal,  cost  or  price  may  play  a  dominant  role  in  source  selection.  The  less  definitive 
the  requirement,  the  more  development  work  required,  or  the  greater  the  performance 
risk,  the  more  technical  or  past  performance  considerations  may  play  a  dominant  role  in 
source  selection. 

While  the  FAR  provides  contracting  officials  with  direction  on  the  use  of  past 
performance  as  an  evaluation  criterion,  the  FAR  provides  very  little  guidance  on  how 
contracting  officials  are  to  evaluate  past  performance.  In  the  absence  of  any  standardized 
process  or  norms,  the  evaluation  of  past  performance  may  become  subjective  in  nature. 

C.  RESEARCH  OBJECTIVE 

The  primary  objective  of  this  research  is  to  develop  a  managerial  model  that  will 
aid  contracting  officials  in  their  efforts  to  more  effectively  implement  the  collection  and 
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use  of  past  performance  information.  The  model  will  be  developed  after  reviewing  the 
current  policies  and  best  practices  guides  for  the  collection  and  evaluation  of  past 
performance  information,  and  then  categorizing  and  analyzing  the  past  performance 
protest  decisions  handed  down  by  the  Comptroller  General  from  October  1,  1997,  to  June 
30,  2000.  Following  the  analysis  of  these  cases,  the  overall  best  practices  and  case 
principles  will  be  linked  with  the  phases  of  the  Federal  acquisition  process. 

D.  RESEARCH  QUESTIONS 

1.  Primary  Research  Question 

What  will  an  analysis  of  published  best  practices  guides  and  Comptroller  General 
case  decisions  suggest  are  the  general  principles  and  best  overall  practices  for  effective 
past  performance  information  management,  and  how  can  these  principles  and  best 
practices  be  integrated  into  the  acquisition  and  contracting  processes  of  a  buying  activity? 

2.  Secondary  Research  Questions 

•  How  have  policies  evolved  with  respect  to  past  performance? 

•  What  best  practice  guides  are  available  that  provide  guidance  on  the 
collection  and  evaluation  of  past  performance? 

•  What  are  the  recommended  methods  for  the  evaluation  of  past 
performance? 

•  How  does  industry  view  the  use  of  past  performance  in  source  selections? 

•  Under  what  circumstances  is  an  offeror  likely  to  file  a  protest  over  the  use 
of  past  performance  information  by  a  Federal  acquisition  professional? 

•  What  are  the  discemable  trends  in  the  number  of  sustained  protests 
involving  past  performance  in  recent  years? 

•  What  elements  of  the  acquisition  contracting  process  will  the  Comptroller 
General  review  in  making  his  decisions? 
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•  What  case  principles  can  be  extracted  from  the  Comptroller  General 
decisions  that  would  aid  contracting  officials  throughout  the  acquisition 
process? 

E.  SCOPE 

The  scope  of  this  thesis  will  include: 

•  A  review  of  the  policies  on  the  collection  and  use  of  past  performance. 

•  An  in-depth  review  of  best  practices  guides  with  the  aim  of  extracting  key 
principles. 

•  An  in-depth  analysis  of  the  Comptroller  General’s  decisions  in  cases 
involving  past  performance  to  identify  trends  and  case  principles. 

•  The  development  of  a  managerial  model  that  integrates  the  case  principles 
from  the  Comptroller  General’s  decisions  with  the  acquisition  contracting 
process. 

This  thesis  will  be  limited  to  protests  that  involve  past  performance  as  an  element 
of  the  protest  that  have  occurred  from  October  1,  1997,  to  June  30,  2000.  The  primary 
assumption  in  this  study  is  that  the  reader  is  familiar  with  the  basic  Federal  acquisition 
contracting  process. 

F.  METHODOLOGY 


The  research  for  this  thesis  will  consist  of  the  following  steps: 

•  Conduct  a  literature  search  of  books,  magazine  articles,  studies,  and  other 
library  information  resources  that  deal  with  the  subject  of  past 
performance. 

•  Conduct  a  survey  of  several  major  contractors  to  gain  a  sense  of  industry’s 
views  on  the  use  of  past  performance  by  Government  contracting  officials. 

•  Conduct  a  search  of  the  General  Accounting  Office  database  for  protest 
cases  that  involved  past  performance  as  an  element  of  the  protest. 

•  Identify  trends  or  key  elements  that  will  allow  the  cases  to  be  categorized 
and  analyzed. 

•  Develop  a  managerial  model  that  integrates  the  acquisition  contract 
process  with  principles  extracted  from  best  practice  guides  and  case 
principles  derived  from  the  Comptroller  General’s  rulings. 
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G.  BENEFITS  OF  THE  RESEARCH 


The  collection  and  evaluation  of  past  performance  information  and  the 
determination  of  best  value  are  some  of  the  most  subjective  areas  in  Government 
contracting.  The  lessons  contained  in  the  best  practices  guides  and  the  lessons  learned 
from  the  Comptroller  General’s  decisions  are  applicable  to  all  Federal  Government 
contracting  officials.  The  model  developed  in  this  study  is  intended  to  assist  Government 
contracting  officials  in  more  effectively  incorporating  the  use  of  past  performance  into 
their  acquisition  and  contracting  processes. 

H.  ORGANIZATION  OF  THE  THESIS 

Following  this  introductory  chapter,  Chapter  II  provides  the  current  working 

definition  of  past  performance  information.  The  chapter  then  provides  background 

information  on  the  evolution  of  procurement  policies  with  respect  to  the  collection  and 

evaluation  of  past  performance  information.  It  also  touches  on  some  of  the  current  issues 

surrounding  the  collection  and  use  of  past  performance  information.  Chapters  III  and  IV 

are  the  analytical  heart  of  the  thesis.  Chapter  III  begins  with  a  review  and  comparative 

analysis  of  several  best  practices  guides.  The  intent  is  to  identify  key  best  practices 

principles  for  the  collection  and  use  of  past  performance  that  can  be  applied  to  a  model  of 

the  Federal  procurement  process.  Next,  the  chapter  covers  the  findings  of  an  industry 

survey  conducted  by  the  researcher.  The  findings  are  used  to  gain  a  sense  of  how  industry 

views  the  use  of  past  performance  information  in  Federal  acquisitions.  The  findings  also 

highlight  the  risks  associated  with  the  use  of  past  performance  (e.g.,  under  what 

conditions  is  an  unsuccessful  offeror  more  likely  to  file  a  protest  involving  the  collection 

and  evaluation  of  past  performance?).  Chapter  IV  begins  with  an  overall  study  of  past 
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performance  protests  brought  before  the  Comptroller  General  from  October  1997  through 
June  2000.  Past  performance  protests  are  categorized,  reviewed,  and  analyzed  with  the 
intent  of  identifying  case  principles  as  they  apply  to  the  Federal  acquisition  process. 
Chapter  V  introduces  the  integrated  managerial  model  that  was  developed  during  the 
study.  The  model  links  the  key  best  practices  principles  and  Comptroller  General  case 
principles  with  the  Federal  acquisition  process.  Chapter  VI  provides  the  answers  to  the 
research  questions,  and  it  includes  the  researcher’s  recommendations  for  areas  of  further 
research. 
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II.  BACKGROUND 


A.  PAST  PERFORMANCE:  DEFINITION  AND  CLARIFICATION 

Past  performance  information  is  one  indicator  of  an  offeror's  ability  to 
successfully  complete  Government  contracts. 

Past  performance  information  is  relevant  information,  for  future  source 
selection  purposes,  regarding  a  contractor's  actions  under  previously 
awarded  contracts.  It  includes,  for  example,  the  contractor's  record  of 
conforming  to  contract  requirements  and  to  standards  of  good 
workmanship;  the  contractor's  record  of  forecasting  and  controlling  costs; 
the  contractor's  adherence  to  contract  schedules,  including  the 
administrative  aspects  of  performance;  the  contractor's  history  of 
reasonable  and  cooperative  behavior  and  commitment  to  customer 
satisfaction;  and  generally,  the  contractor's  business-like  concern  for  the 
interest  of  the  customer.  [Ref.  16:42.1501] 

While  knowledge  of  the  working  definition  of  past  performance  is  a  must  for 
acquisition  professionals,  they  must  also  keep  in  mind  the  difference  between 
performance  risk  and  proposal  risk.  Proposal  risk  is  associated  with  an  offeror’s  proposed 
approach  in  meeting  the  Government’s  requirements  and  may  be  integrated  into  the 
analysis  of  each  specific  evaluation  factor  or  sub-factor.  Performance  risk  is  associated 
with  an  offeror’s  likelihood  of  success  in  performing  the  Government’s  requirements  as 
indicated  by  that  offeror’s  record  of  past  performance.  [Ref.  3] 

It  is  also  important  to  recognize  the  difference  between  past  performance  and 
experience.  “Experience  reflects  whether  contractors  have  performed  similar  work 
before.  Past  performance,  on  the  other  hand,  describes  how  well  contractors  have 
performed  the  work.”  [Ref.  13:p.  7]  As  an  example,  a  company  may  have  twenty  years 
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experience  in  producing  an  item,  but  it  can  still  have  a  poor  performance  record  with 
respect  to  cost,  quality,  and  delivery. 

B.  EVOLUTION  OF  POLICIES 

The  Office  of  Federal  Procurement  Policy  (OFPP)  Policy  Letter  92-5  [Ref.  34]  set 
in  motion  the  events  that  have  evolved  into  today’s  more  standardized  approach  by 
Federal  procurement  agencies  for  the  collection,  recording,  and  utilization  of  past 
performance  information.  In  its  letter,  OFPP  recognized  the  importance  of  past 
performance  information  as  a  tool  to  aid  in  the  source  selection  process  and  to  “  improve 
contractor  performance.”  Two  major  requirements  that  were  levied  on  Federal  agencies 
were: 

•  Newly  established  firms  were  to  be  allowed  to  compete  for  contracts  even 
though  they  lacked  a  history  of  past  performance 

•  Past  performance  was  to  be  used  as  an  evaluation  factor  in  all 
competitively  negotiated  contracts  expected  to  exceed  $100,000  [Ref.  34: 
pp.  2-3] 

OFPP  also  recognized  that  guiding  requirements  were  necessary  to  ensure 
integrity  and  fairness.  OFPP  looked  to  the  FAR  Council  for  implementation  of  these 
policies.  OFPP  mandated  that  the  policy  changes  addressed  in  OFPP  Policy  Letter  92-5 
were  to  be  incorporated  into  the  Federal  Acquisition  Regulation  within  210  days.  [Ref. 
34:p.  5]  However,  the  Federal  Acquisition  Streamlining  Act  (FASA)  of  1994  made  the 
OFPP  requirements  a  law,  and  the  FAR  Council  released  Federal  Acquisition  Circular 
(FAC)  90-26.  This  Circular  became  effective  on  May  30,  1995,  and  it  stated  that  past 
performance  must  be  a  mandatory  evaluation  factor  for  all  solicitations  with  an  estimated 
value  of: 
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•  $1,000,000  issued  on  or  after  July  1, 1995 

•  $500,000  issued  on  or  after  July  1,  1997 

•  $1 00,000  issued  on  or  after  January  1 , 1 999 

After  almost  eight  years,  through  various  rewrites  to  the  Federal  Acquisition 
Regulation,  the  policies  envisioned  in  OFPP  92-5  have  been  met.  The  Federal 
Acquisition  Regulation’s  Statement  of  Guiding  Principles  now  reflects  this  emphasis  on 
the  use  of  past  performance  by  stating  that: 

The  Federal  Acquisition  System  will  satisfy  the  needs  of  its  customers  in 
terms  of  cost,  quality  and  timeliness  by. . .  using  contractors  who  have  a 
track  record  of  successful  past  performance  or  who  demonstrate  a  current 
superior  ability  to  perform.  [Ref.  16:1.102/102-1] 

When  FAC  97-02  implementing  the  FAR  Part  15  (Contracting  by  Negotiation) 
rewrite  was  issued  on  September  30,  1997,  it  placed  several  requirements  on  Federal 
acquisition  professionals.  FAR  Part  15  now  requires  that  past  performance  information 
shall  be  evaluated  in  all  source  selections  for  negotiated  competitive  acquisitions  issued 
on  or  after  January  1,  1999,  for  acquisitions  expected  to  exceed  $100,000  unless  the 
contracting  officer  documents  the  reasons  why  past  performance  is  not  an  appropriate 
evaluation  factor.  FAR  Part  42  (Contract  Administration  and  Audit  Services)  also 
requires  that  past  performance  information  is  to  be  collected  and  retained  for  a  period  of 
three  years  for  all  acquisitions  over  $100,000. 

C.  RECENT  DEVELOPMENTS 

In  December  1999,  the  Department  of  Defense  released  a  report  on  Price-Based 
Acquisition  (PBA).  The  report  was  in  response  to  Congress’  direction  in  Section  912(c) 
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of  The  National  Defense  Authorization  Act  of  Fiscal  Year  1998,  which  directed  the 
Department  of  Defense  to  identify  additional  key  steps  that  DoD  could  and  should  take  to 
build  upon  and  enhance  previous  acquisition  reforms.  The  committee  on  PBA  looked  at 
initiatives  that  would  enhance  the  use  of  price-based  procurements  in  Federal  programs. 
PBA  reduces  the  burden  on  prospective  offerors  by  relieving  them  of  the  requirement  to 
provide  certified  cost  and  pricing  data.  The  report  recommended  numerous  initiatives  to 
continue  the  current  Revolution  in  Business  Affairs.  With  regard  to  past  performance  the 
report  recommended  that: 

Past  performance  be,  as  a  matter  of  Department-wide  policy,  at  least  equal 
to  the  highest  ranking  factor  in  every  DoD  source  selection  valued  at  more 
than  $1  million,  unless  waived  by  the  Head  of  the  Contracting  Activity 
(HCA)  for  a  specific  exception.  [Ref.  36:p.  PS-7] 

The  committee  that  drafted  the  report  viewed  this  possible  policy  change  as  an 
enabler  for  expanding  the  use  of  PBA,  and  encouraging  offerors  to  be  more  realistic  in 
their  promises  and  pricing. 

On  June  2,  2000,  The  Honorable  Jacques  S.  Gansler,  the  Under  Secretary  of 
Defense  for  Acquisition,  Technology,  and  Logistics  released  the  report  titled  “The  Road 
Ahead.”  As  an  element  of  the  continuing  Revolution  in  Business  Affairs,  the  paper 
points  out  that  the  “Department  of  Defense  will  increasingly  rely  on  an  integrated  civil- 
military  industrial  base  in  lieu  of  a  defense-unique  industrial  base.”  [Ref.  44:p.  11]  One 
of  the  steps  for  achieving  the  goal  of  an  integrated  industrial  base  is  the  establishment  of 
“strategic  alliance  relationships  among  buying  commands  and  suppliers.”  [Ref.  44  :p.  11] 
The  expectation  is  that  Federal  procurement  agencies  will  gain  greater  insight  into  the 
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commercial  world’s  best  practices,  one  of  which  is  the  use  of  past  performance  in  source 
selections. 

Recently  two  articles  have  appeared  that  indicate  that  over  the  last  five  years, 
since  the  requirements  for  the  collection  and  use  of  past  performance  information  have 
become  mandatory.  Federal  procurement  agencies  have  not  successfully  institutionalized 
its  use.  The  first  article  appeared  in  the  journal  Contract  Management.  In  his  article  “Past 
Performance  is  Not  Working,”  Joseph  Beausoleil  points  out  that  less  than  20  percent  of 
the  contracts  that  were  completed  in  1998  and  1999  were  evaluated  in  a  way  that  met  the 
FAR  requirements.  [Ref.  4:p.  29]  This  article  maintains  that  the  current  process  for  the 
post-award  collection  and  evaluation  of  contractor  performance  is  too  cumbersome  in 
today’s  Federal  acquisition  environment,  and  for  the  process  to  be  effective  it  must  be 
streamlined. 

The  second  article  appeared  in  the  Philadelphia  Inquirer  on  July  24,  2000.  In  their 
article  “Firms  That  Defrauded  U.S.  Still  Getting  Government  Contracts,”  John  Solomon 
and  Katherine  Pfleger  identified  1 ,020  companies  that  were  sued  or  prosecuted  for  fraud 
over  the  last  five  years.  By  checking  the  master  list  of  companies  that  are  barred  from 
Federal  business,  they  found  that  737  of  these  companies  were  still  eligible  for  future 
contracts.  [Ref.  37]  In  their  conclusion,  they  point  out  that  several  of  these  companies 
have  subsequently  won  contract  awards.  The  article  paints  a  picture  of  a  procurement 
process  that  places  little  value  on  an  offeror’s  past  performance. 
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D.  SUMMARY 


The  tone  of  these  two  articles  is  that  the  Federal  acquisition  workforce  is 
struggling  with  the  requirements  for  the  collection  and  evaluation  of  past  performance 
information.  This  view  is  supported  by  a  1999  General  Accounting  Office  Report  “Best 
Practices:  DoD  Training  Can  Do  More  to  Help  Weapons  Systems  Programs  Implement 
Best  Practices.”  GAO  investigated  the  implementation  of  five  current  best  practices  (past 
performance,  cost  as  an  independent  variable,  performance  specifications,  integrated 
product  teams,  and  supplier  relationships)  at  six  program  offices.  None  of  the  six 
program  offices  was  considered  a  leader  in  all  five  practices,  but  they  were  chosen 
because  they  represented  best-case  examples  for  one  or  more  of  the  specific  practices. 

The  report  points  out  that  over  the  last  five  years,  “over  40  initiatives  have  been 
introduced  to  the  DoD  acquisition  work  force,  without  delineating  which  are  the  most 
important  for  weapons  systems  programs.”  [Ref.  6:p.  59]  The  report  further  notes  that: 

The  Defense  Acquisition  University  incorporates  best  practice  topics  into 

DAWIA  courses  as  drop  in  modules  that  provide  a  survey  of  the  topic. 

The  information  conveyed  is  enough  to  provide  a  general  awareness  of  the 

concept  but  not  enough  to  implement  the  practice  at  the  workplace.  [Ref. 

6:p.  45] 

To  augment  the  training  that  has  been  available,  numerous  agencies  have 
published  best  practices  guides  to  assist  Federal  procurement  professionals  in 
understanding  not  only  the  requirements  for  the  collection  and  use  of  past  performance 
information,  but  also  how  to  go  about  implementing  the  requirements  into  their  day  to 
day  operations. 
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In  the  eight  years  since  the  release  of  OFPP  Policy  Letter  92-5,  Federal 
procurement  personnel  have  had  to  wrestle  with  the  requirements  for  the  collection  and 
use  of  past  performance  information,  even  as  they  have  seen  policy  makers  placing 
increased  emphasis  on  its  use  in  source  selections.  The  regulatory  requirements  for  the 
collection  and  use  of  past  performance  information  are  now  statutory  requirements  that 
are  codified  in  the  Federal  Acquisition  Regulation.  If,  as  Dr.  Gansler  notes,  “the  road 
ahead  is  clear,”  [Ref.  44]  then  these  requirements  will  become  more  stringent  as  the 
Federal  acquisition  community  adopts  more  of  the  Commercial  Market’s  best  business 
practices. 
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III.  BEST  PRACTICES  AND  A  SENSE  OF  INDUSTRY 


A.  INTRODUCTION 

This  chapter  reviews  five  best  practices  guides  with  the  goal  of  identifying  the 
key  principles  for  applying  best  practices  associated  with  past  performance  information. 
It  then  reports  the  results  of  a  survey  of  industry  representatives,  to  gain  a  sense  of  how 
industry  views  the  use  of  past  performance  information  by  Government  acquisition 
professionals. 

B.  BEST  PRACTICES  GUIDES  AND  BEST  PRACTICES 

For  the  purposes  of  this  thesis,  a  best  practice  is  defined  as  a  method  of 
accomplishing  an  acquisition  function  or  process  that  is  considered  to  be  superior  to  other 
methods.  Best  practices  are  developed  through  experience  and  lessons  learned,  and  when 
successfully  identified  and  applied,  best  practices  can  reduce  costs  and  improve 
organizational  efficiency. 

Since  1995  numerous  best  practices  guides  and  handbooks  have  been  developed 
to  assist  the  Government  acquisition  workforce  implement  policies  for  the  collection  and 
use  of  past  performance  information. 

Five  guides  were  reviewed: 

•  Best  Practices  for  Collecting  and  Using  Current  and  Past  Performance 
Information,  Office  of  Federal  Procurement  Policy,  1995  &  2000 

•  U.S.  Marine  Corps  Best  Value  Source  Selection  Handbook,  Marine  Corps 
Systems  Command,  1 997 
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Army  Material  Command  (AMC)  Pamphlet  715-3,  Contracting  for  Best 
Value:  A  Best  Practices  Guide  to  Source  Selection.  Army  Material 
Command,  1998 

A  Guide  to  Collection  and  Use  of  Past  Performance  Information, 
Department  of  Defense,  1999 

Source  Selection  Activities  Guide  (Performance  Risk  Assessment  Group 
(PRAG))  Guide,  Air  Force  Material  Command,  2000 


The  review  focused  on:  how  relevant  each  guide  is  to  the  current  acquisition 
environment,  how  reliable  and  up-to-date  the  information  contained  in  the  guide  is,  and 
how  responsive  the  guide  is  to  the  needs  of  the  acquisition  workforce.  Plainly  put,  the 
criteria  for  reviewing  these  guides  were: 

•  How  clearly  is  the  information  presented? 

•  How  current  or  up-to-date  is  the  information  contained  in  the  guide? 

•  What  phases  of  the  acquisition  process  does  the  guide  address? 

•  Does  the  guide  provide  information  that  can  be  used  in  day-to-day 
operations? 

Alone,  each  of  these  guides  provides  valuable  guidance  for  the  use  of  past 
performance  information  (PPI).  Together  they  identify  a  comprehensive  list  of  key  best 
practices  for  the  collection  and  use  of  PPI  throughout  the  entire  Federal  acquisition 
process. 

1.  Best  Practices  &  Best  Value  Guides 

The  Office  of  Federal  Procurement  Policy’s  Best  Practices  Guide  was  originally 
released  as  an  interim  guide  in  May  1995,  and  the  final  edition  was  released  in  May  2000. 
It  is  a  well-written  thirty-two-page  document  that  presents  techniques  and  practices  for 
working  level  acquisition  professionals.  The  goal  of  the  guide  is  to  assist  Federal 
acquisition  agencies  in  reaping  the  benefits  of: 
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•  Better  contract  performance  because  of  dialog  between  the  contractor  and 
the  Government 

•  Better  ability  to  select  high  quality  contractors  for  new  contracts,  because 
contractors  know  the  assessments  will  be  used  in  future  award  decisions 
[Ref.  5:p.  2] 

Its  main  focus  is  on  purchases  above  the  simplified  acquisition  threshold  (SAT)  of 
$100,000.  However,  it  points  out  that  FAR  Part  13  allows  Contracting  Officers  to  use 
whatever  information  is  available  to  them  about  an  offeror’s  past  performance  to  make 
award  decisions. 

The  guide  has  only  three  chapters:  Introduction,  Evaluating  and  Recording 
Current  Performance,  and  Using  Current  and  Past  Performance  as  a  Source  Selection 
Factor.  Each  chapter  provides  tips,  hints,  and  examples  for  use  of  best  practices.  This 
guide  is  a  good  starting  point  for  anyone  interested  in  how  to  collect  and  use  PPI. 

The  Marine  Corps’  Best  Value  Handbook  was  released  in  1997,  and  its  stated 
purpose  is: 

To  provide  guidance  and  assist  Marine  Corps  Systems  Command 

acquisition  personnel  in  understanding  the  procedures  of  the  source 

selection  process  for  competitively  negotiated  acquisitions  where  selection 

is  based  on  technical  and  other  factors  in  addition  to  price.  [Ref.  45 :p.  4] 

The  handbook  covers  the  elements  of  the  Federal  acquisition  process  from  the 
formation  of  the  Acquisition  Team  through  the  debriefing  of  the  unsuccessful  offerors.  It 
is  rather  lengthy  (fifty-four  pages),  and  it  is  broken-down  into  eight  chapters: 

•  General  Information 

•  Roles  &  Responsibilities 

•  Source  Selection  Plan 

•  Best  Value  Procurement 
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•  Conducting  the  Evaluation 

•  Request  for  Proposals 

•  Post-Evaluation  Actions 

•  Debriefing 

While  the  format  of  the  handbook  is  straightforward,  the  writing  style  is  not  as 
smooth  as  other  guides,  and  at  times  it  contradicts  itself.  For  example,  in  Chapter  III 
under  the  heading  of  Rating  Systems,  it  states  that  numerical  ratings  with  narratives  are 
the  most  useful  method  for  the  evaluation  and  the  documentation  of  past  performance. 
[Ref.  45  :p.  14]  Then  in  Chapter  IV  under  Lessons  Learned,  it  points  out  “the  formula 
approach  to  best  value  has  fallen  into  disfavor  because  it  may  lock  the  selection  official 
to  the  highest  scored  proposal....”  [Ref.  45 :p.  26] 

Another  shortfall  of  the  handbook  is  that  it  has  not  kept  up  with  current  policy 
changes.  Most  notably,  it  has  not  been  updated  to  incorporate  the  requirements  of  the 
FAR  Part  15  rewrite  released  in  1997.  While  past  performance  has  been  a  mandatory 
evaluation  factor  for  all  negotiated  competitive  acquisitions  over  $100,000  since  January 
1,  1999,  the  handbook  advises  Marine  acquisition  professionals  that  “The  Government 
must  always  take  into  consideration  the  mandatory  factors:  cost  and  price,  and  soon  past 
performance.”  [Ref.  45  :p.  26] 

Despite  its  noticeable  flaws,  this  handbook  still  provides  useful  techniques  and 
practices  for  the  collection  and  use  of  PPI.  With  editing  and  updating  to  incorporate  the 
latest  acquisition  reform  initiatives,  this  handbook  would  be  one  of  the  most 
comprehensive  guides  available. 
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AMC  Pamphlet  715-3,  Contracting  for  Best  Value:  A  Best  Practices  Guide  to 
Source  Selection  was  published  in  January  1998.  The  guide  was  designed  “for  use  by  the 
entire  acquisition  workforce  to  promote  consistent  understanding  of  best  value  and  the 
various  processes  and  techniques  that  can  be  used  to  achieve  it.”  [Ref.  2:p.  1]  Prior  to  its 
release  the  guide  was  updated  to  reflect  the  latest  acquisition  reform  initiatives,  most 
notably  the  rewrite  of  FAR  Part  15.  The  researcher  found  this  guide  to  be  the  most 
comprehensive  in  terms  of  scope  of  applicability.  It  is  composed  of  ten  sections: 

•  Overview 

•  Planning  for  Source  Selection 

•  Drafting  the  Source  Selection/Evaluation  Plan 

•  Solicitation 

•  Evaluation  Considerations 

•  Selection  Decision 

•  Notification  and  Debriefing  of  Unsuccessful  Offerors 

•  Lessons  Learned 

•  Top  Ten  Messages 

•  Appendices 

While  AMC  pamphlet  715-3  is  long  in  comparison  to  the  other  guides  reviewed, 
it  has  a  flow  that  makes  it  easy  to  read  and  understand.  Throughout  the  guide,  key  points 
and  ideas  are  centered  and  highlighted  to  draw  the  reader’s  attention.  Overall,  the 
researcher  found  this  guide  to  be  the  most  useful  best  practices  guide  available  to  the 
acquisition  workforce  for  the  collection  and  use  of  PPL 

The  Department  of  Defense’s  “A  Guide  to  Collection  and  Use  of  Past 
Performance  Information”  was  released  in  May  1999.  It  is  intended  for  use  by  the  entire 
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acquisition  workforce  in  both  Government  and  industry  to  promote  the  goal  of  achieving 
“best  value.”  [Ref.  13:p.  ii]  The  guide  is  broken  down  into  four  sections: 

•  Introduction 

•  Obtaining  Past  Performance  Information 

•  Use  of  Past  Performance  Information  in  Source  Selection 

•  Appendices 

The  guide  highlights  best  practices  for  the  use  of  PPI  during  source  selections  and 
the  collection  of  PPI  during  post-award  performance.  Like  the  OFPP  guide,  it  is  a 
relatively  short  document;  it  is  composed  of  sixteen  pages  of  text  and  nineteen  pages  of 
appendices  that  cover  areas  such  as  definitions,  references,  and  common  DoD  assessment 
rating  schemes.  The  writing  is  clear  and  well  structured.  Key  concepts  are  highlighted 
and  offset.  Again  like  the  OFPP  guide,  it  is  a  good  starting  point  for  anyone  who  is 
interested  in  learning  how  to  collect  and  use  past  performance  information  throughout  the 
Federal  acquisition  process. 

While  the  previous  guides  focused  on  the  best  practices  for  the  collection  and  use 
of  past  performance  information  by  the  overall  acquisition  team  or  workforce,  the  Air 
Force’s  Source  Selection  Activities  Guide  focuses  on  the  actions  of  the  Performance  Risk 
Assessment  Group  (PRAG).  The  PRAG  is  one  of  the  teams  of  the  Source  Selection 
Evaluation  Board  (SSEB).  It  is  the  team  within  the  source  selection  organization  that  is 
tasked  with  assessing  the  relevant  past  and  present  performance  information  and 
assigning  a  performance  confidence  rating.  [Ref.  38:p.  2]  The  Source  Selection  Activities 
Guide  is  a  short  fourteen-page  document  that  covers  the  steps  for  the  collection  and  use 
of  past  performance  information  from  prior  to  release  of  the  request  for  proposals  to  the 
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documentation  and  briefing  of  the  PRAG’s  analysis.  [Ref.  38:p.  1]  The  guide  has  five 
major  sections: 

•  PRAG  Source  Selection  Guide:  Introduction 

•  Prior  to  Release  of  Requests  for  Proposals  (RFP) 

•  Prior  to  Receipt  of  Proposals 

•  After  Receipt  of  Proposals 

•  After  Analysis  of  Data 

The  information  presented  in  each  section  flows  smoothly  from  one  key  point  to 
another.  However,  one  drawback  is  that  the  drafters  of  the  guide  use  several  Service- 
unique  acquisition  terms  that  will  be  unfamiliar  to  the  non-Air  Force  reader  (e.g.,  the 
Procuring  Contracting  Officer  (PCO)  is  called  the  Source  Selection  Focal  Point,  and  past 
performance  sub-factors  are  called  Mission  Capability  sub-factors). 

The  review  of  these  guides  suggests  that  to  remain  effective,  they  must  be  living 
documents.  While  it  may  be  difficult  to  keep  them  current  with  the  rapidly  changing 
acquisition  environment,  the  stakeholders  of  these  guides  must  invest  the  time  and  energy 
to  keep  them  relevant  to  the  current  acquisition  environment  and  responsive  to  the  needs 
of  the  acquisition  professionals  who  use  them.  While  each  guide  presented  a  slightly 
different  perspective  on  the  use  and  collection  of  PPI,  there  are  best  practices  that 
transcend  each  Service  or  Department. 

2.  Best  Practices 

In  developing  a  comprehensive  list  of  best  practices  (BP)  for  the  collection  and 

use  of  PPI,  the  researcher  examined  the  BPs  presented  in  each  guide.  The  first  criterion 

for  making  the  list  was  consensus  among  the  guides.  Not  every  guide  covered  the  same 

phases  of  the  acquisition  process,  but  there  was  sufficient  overlap,  so  that  the  researcher 
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found  that  most  of  the  guides  provided  clarification,  justification,  and  amplification  for 
the  use  of  a  BP.  The  second  criterion  for  making  the  list  was  the  prudent  businessman 
concept.  When  a  BP  was  addressed  in  only  one  guide  the  researcher  made  the  judgment 
call  on  whether  a  prudent  businessman  would  make  use  of  the  practice.  The  researcher 
had  to  ask  himself  the  question,  “Did  the  practice  make  sense,  and  what  were  the  benefits 
of  using  the  practice?”  By  combining  BPs  from  each  of  the  guides,  the  researcher  has 
composed  a  list  of  what  he  judges  to  be  the  top  eleven  BP’s  for  the  collection  and  use  of 
PPI.  The  list  is  presented  here  and  it  is  included  as  Appendix  “A”  for  easier  reference 
later. 

•  (BP1).  Tailor  the  use  and  evaluation  of  past  performance  information  to 
fit  the  needs  of  the  acquisition. 

•  Weight  past  performance  to  ensure  that  it  is  a  valid  discriminator 

•  Limit  sub-factors  to  true  discriminators.  Recommended  sub¬ 
factors  are: 

•  Quality  of  performance 

•  Cost  performance 

•  Schedule  performance 

•  Business  Relations 

•  (BP2)  Invest  in  command  or  program  resources  needed  for  a  competent 
and  well  documented  best  value  source  selection. 

•  Include  the  source  selection  authority  as  an  active  participant 

•  Train  all  the  evaluators  in  best  practices 

•  (BP3)  Conduct  pre-solicitation  exchanges  of  information  with  industry  to 
explain  the  approach  you  will  use  to  evaluate  performance  risk,  (e.g.,  draft 
solicitations  &  pre-solicitation  conferences) 

•  Pre-solicitation  exchanges  can  help  to  ensure  that  potential  offerors 
have  a  clear  understanding  of  how  past  performance  will  be 
evaluated 
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•  (BP4)  Structure  the  solicitation  to  communicate  effectively  to  potential 
offerors. 

•  Encourage  new  entrants  to  the  market  to  identify  other  related  PPI 

•  (BP5)  Use  the  most  relevant,  recent  PPI  available  in  making  the  source 
selection  decision.  PPI  can  come  from  a  variety  of  sources. 

•  Other  Federal  Agencies  and  Government  databases 

•  State  and  Local  Governments 

•  Commercial  contractors 

•  References  provided  by  the  offeror 

•  Quality  certificates  and  awards 

•  (BP6)  Conduct  reference  checks  and  look  for  patterns  or  trends. 
Whenever  possible  request  two  points  of  contact  (POC)  for  each 
reference.  Use: 

•  Questionnaires 

•  Face  to  face  interviews.  Keep  in  mind  that  these  provide  more 

insight  than  other  methods;  they  can  expose  problems  with  the 
contractor  that  can  be  discussed  during  negotiations 

•  Telephone  interviews 

•  Some  combination  of  the  above  methods 

•  (BP7)  Document  strengths,  weaknesses,  and  risks  of  each  proposal  to 
support  the  cost/past  performance  tradeoff. 

•  Recognize  that  numerical  rating  systems  can  imply  false  precision 
and  limit  flexibility  in  the  decision  making  process 

•  (BP8)  Justify  price  premiums  with  tradeoff  documentation  regardless  of 
the  selected  proposal’s  cost  or  past  performance  superiority. 

•  The  greater  the  price  premium,  the  greater  the  documentation  must 
show  the  worth 

•  (BP9)  Ensure  that  the  source  selection  decision  is  consistent  with  the 
relative  weights  assigned  to  the  evaluation  factors  in  the  solicitation. 

•  (BP  10)  Conduct  a  proper  and  timely  debriefing. 

•  Debriefings  provide  unsuccessful  offerors  with  the  opportunity  to 
learn  about  their  strengths  and  weaknesses  and  how  to  improve 
future  proposals  submitted  to  the  Government 

•  Offerors  rely  heavily  on  information  provided  during  debriefing 
sessions  to  influence  their  decisions  regarding  filing  protests 
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•  (BP  11)  Assess  current  contract  performance;  it  is  a  basic  “best  practice” 

for  good  contract  administration,  and  it  is  one  of  the  most  important  tools 
available  for  ensuring  good  contractor  performance. 

By  adhering  to  this  comprehensive  listing  of  best  practices,  acquisition 
professionals  should  be  able  to  more  effectively  implement  the  use  of  PPI  into  their 
acquisition  processes.  One  positive  side-effect  of  increasing  the  effectiveness  of  the  use 
of  PPI  is  that  Federal  acquisition  professionals  will  present  a  more  level  playing  field  to 
businesses  that  provide  products  and  services  to  the  Federal  Government.  This  should  in 
turn  mitigate  the  risk  of  successful  (sustained)  protests  brought  about  by  unsuccessful 
offerors. 

C.  THE  SENSE  OF  INDUSTRY 

The  purpose  of  this  section  is  to  gain  an  appreciation  for  how  industry  is  reacting 
to  the  increased  use  of  past  performance  information  by  Government  procurement 
officials.  This  section  will  focus  on  pricing  risk.  For  the  purposes  of  this  study  pricing 
risk  will  be  defined  as  how  companies  may  potentially  modify  their  pricing  strategies 
dependent  upon  the  weights  assigned  to  past  performance  and  cost  or  price  (e.g.,  will  the 
companies  accept  more  risk  by  lowering  profits  to  win  a  contract?). 

1.  Hypothesis 

The  researcher’s  hypothesis  is  that  companies  will  modify  their  pricing  strategies 
when  past  performance  is  rated  significantly  higher  than  cost  or  price,  and  that  past 
performance  will  have  no  effect  on  pricing  strategies  when  past  performance  is  rated 
significantly  less  than  cost  or  price. 
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The  hypothesis  is  based  upon  the  following  assumptions: 

•  Higher  priced  proposals  will  result  when  past  performance  is  heavily 
weighted  in  the  solicitation,  and  a  company  believes  that  its  past 
performance  will  be  rated  above  its  competition 

•  Lower  priced  proposals  will  result  when  past  performance  is  heavily 
weighted  in  the  solicitation,  and  a  company  believes  that  its  past 
performance  will  be  rated  below  its  competition 

•  Pricing  strategies  will  be  unaffected  when  past  performance  is  weighted 
significantly  less  than  cost  or  price 

Initially,  ten  companies  were  contacted  to  test  the  validity  of  the  hypothesis. 
Ultimately,  contracting  officials  from  five  of  these  companies  agreed  to  participate  in  the 
survey,  and  they  were  able  to  respond  to  a  set  of  research  questions.  The  following 
question  was  developed  and  sent  to  the  five  companies:  (The  question  is  also  reproduced 
as  Appendix  “B”). 

How  might  the  following  scenarios  affect  your  pricing  strategy? 

a.  You  Have  Received  a  Request  for  Proposal  (RFP)  that  Indicates 
that  Past  Performance  will  be  Weighted  Significantly  more  than 
Cost  or  Price  (Past  Performance  50%,  Cost  or  Price  20%,  and 
Technical  30%) 

(1)  In  the  first  case,  you  are  confident  that  your  past 

performance  rating  will  be  higher  than  your  competitors. 

(2)  In  the  second  case,  you  feel  that  your  past  performance 
rating  will  be  somewhat  less  than  your  competitors. 

b.  You  Have  Received  an  RFP  that  Indicates  that  Past  Performance 
will  be  Weighted  Significantly  less  than  Cost  or  Price  (Cost  or 
Price  70%,  Past  Performance  10%,  and  Technical  20%). 

(1)  In  the  first  case,  you  are  confident  that  your  past 

performance  rating  will  be  higher  than  your  competitors. 
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(2)  In  the  second  case,  you  feel  that  your  past  performance 
rating  will  be  somewhat  less  than  your  competitors. 

2.  Analysis  of  Responses  to  the  Research  Questions 

a.  When  Past  Performance  is  Rated  Significantly  More  than  Cost 
or  Price 

This  area  generated  the  strongest  responses  and  the  most  feedback  from 
the  respondents.  When  asked  about  the  effects  of  past  performance  on  pricing  strategies 
when  the  offeror  was  confident  that  their  performance  rating  would  be  higher  than  their 
competitors,  three  industry  officials  responded  that  past  performance  would  have  little  or 
no  effect  on  their  pricing  strategies.  Some  of  their  responses  were: 

•  We  would  want  to  shoot  for  a  competitive  price  [Ref.  4 1  ] 

•  The  only  area  that  would  justify  a  higher  cost  or  price  is  in  the  area  of 
technical  performance  [Ref.  40] 

•  The  percentage  weight  that  the  source  selection  committee  would  assign  to 
past  performance  whether  it’s  50%  or  10%  has  no  impact  on  our  pricing 
strategy.  The  question  is  really  what  is  best  for  our  business... we 
generally  wouldn’t  change  our  pricing  strategies....  We  wouldn’t 
artificially  raise  our  costs....  We  have  a  standard  proposal  process  and  a 
forward  pricing  rate  agreement  (FPRA).  [Ref.  42] 

The  other  two  respondents  indicated  that  their  pricing  strategies  might 
actually  be  to  lower  the  contract  price  for  the  following  reasons: 

•  If  past  performance  was  weighted  heavily  and  we  were  confident  that  our 
rating  would  be  higher. . .  we  would  accept  less  risk  in  our  pricing. .  .but  we 
would  not  bid  a  price  that  we  would  lose  money  on  [Ref.  43] 

•  Since  you  wish  to  have  an  ongoing  relationship  with  your  customer  you 
will  not  gouge  him  because  you  think  that  you  have  an  edge  in  past 
performance  [Ref.  43] 

•  The  pricing  strategy  would  have  to  take  into  account  an  assessment  of  all 
the  factors. .  .in  arriving  at  a  “price  to  win”  [Ref.  39] 
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The  common  thread  winding  through  all  of  these  responses  is  that  when 
past  performance  is  weighted  heavily,  a  Government  contracting  officer  may  not  see 
inflated  prices  from  those  companies  that  are  potentially  the  leaders  in  past  performance. 
On  the  contrary,  the  contracting  officer  may  actually  see  deflated  prices. 

A  suspicion  of  the  motives  for  the  use  of  PPI  by  Government  procurement 
officials  or  a  misunderstanding  of  the  evaluation  process  by  offerors  are  two  of  the  major 
reasons  why  prices  may  be  kept  at  or  below  the  industry  average.  While  the  source 
selection  criteria  are  spelled  out  in  the  request  for  proposals  (RFP),  several  of  the 
respondents  indicated  that: 

•  To  us  cost  or  price  is  always  important  (if  for  no  other  reason  than  the 
impact  it  has  on  the  customer’s  mentality)  [Ref.  40] 

•  You  can’t  be  sure  how  the  evaluators  will  apply  the  criteria  [Ref.  43] 

•  Given  the  methods  for  accumulating  PPI  and  the  subjective  nature  of  the 
selection  process  no  procurement  is  absolute  [Ref.  39] 

These  responses  are  echoed  in  a  1997  survey  conducted  by  the  Society  of 

American  Military  Engineers  (SAME).  The  survey  was  designed  to  evaluate  the  fairness 

of  the  DoD  performance  evaluation  system.  Below  are  two  of  the  findings  that  were 

reported  by  Neil  B.  Clipsham  “Appraising  Performance  Appraisal  Systems”  The  Military 

Engineer,  (April-May  1998):  31. 

•  75%  of  Contractors,  Consultants,  and  Suppliers  believe  the  Federal 
Acquisition  Regulation  (FAR)  and  directives  allow  too  much  latitude  to 
evaluators,  resulting  in  widely  differing  weighting  and  selection  of 
evaluation  factors. . . 

•  50%  responded  that  changes  are  needed  to  ensure  a  stable  standard  for 
measuring  performance  [Ref.  9] 

When  responding  to  the  question  that  dealt  with  the  situation:  when  past 

performance  is  rated  highly  and  your  past  performance  rating  may  be  somewhat  less  than 
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your  competitors,  there  appears  to  be  multiple  approaches.  The  first  approach  indicates 
that  there  is  a  strong  possibility  some  companies  would  look  at  lowering  their  profit/fee 
or  price  in  an  attempt  to  win  the  contract  award.  Two  of  the  reasons  that  were  given  for 
this  approach  are: 

•  We  may  make  a  business  decision  to  lower  our  fee  and  accept  a  cost 
challenge  because  obtaining  the  work  is  critical  to  our  business  goals  [Ref 
42] 

•  If  we  felt  strongly  about  the  contract,  we  may  sharpen  our  pencil  on  price 
or  reduce  our  fee  or  profit  [Ref.  41] 

This  approach  seems  to  reflect  the  actions  of  companies  that  are  hungry 
for  work.  If  the  companies  are  in  critical  need  of  the  work,  then  lowering  the  profit/fee  or 
price  may  be  the  only  way  that  the  companies  can  competitively  bid  for  the  contracts. 

However,  the  approaches  that  two  of  the  other  companies  proposed  seem 
to  be  more  effective  from  the  standpoint  of  good  business  sense.  The  representative  from 
Lockheed  Martin  stated  that  he  would  attempt  to  draw  attention  to  “relevant”  past 
performance  to  mitigate  the  effects  of  any  negative  past  performance.  Under  his 
approach,  he  would  use  the  “best  value”  for  his  company’s  technical  approach  as  the 
benchmark  for  his  pricing  strategy.  [Ref.  39] 

The  representative  from  Northrop-Grumman  takes  the  approach  that 
“nothing  can  offset  bad  past  performance,  not  even  lowering  cost  or  price.”  He  proposed 
that  to  be  able  to  competitively  bid  for  contracts,  his  company  would  place  “emphasis  on 
the  good  for  obvious  reasons  and  emphasis  on  the  bad  to  prove  that  they  were  aware  of 
the  problem  and  that  they  had  a  plan  in  place  to  fix  it.”  To  put  teeth  into  the  corrective 
action  plan,  he  indicated  that  his  company  would  also  develop  a  “negative  incentives 
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program”  that  would  be  tied  to  the  corrective  action  plan.  From  the  researcher’s 
viewpoint,  of  the  three  approaches  proposed  by  the  respondents,  this  approach  represents 
the  best  win/win  scenario.  The  company  is  internally  motivated  by  the  self-imposed 
negative  incentives  plan  to  take  corrective  action,  thereby  reducing  the  overall  contract 
risk  to  the  Government. 

b.  When  Past  Performance  is  Rated  Significantly  Less  than  Cost  or 
Price 

Here  the  respondents  answered  overwhelming  that  their  companies  would 
not  modify  their  pricing  strategies.  Nearly  every  respondent  indicted  that  their  company 
would  base  its  pricing  strategy  on  cost  or  price  factors.  It  did  not  matter  if  they  believed 
that  their  company’s  past  performance  was  rated  higher  or  lower  than  their  competitors. 

From  a  business  standpoint  these  responses  are  clearly  understandable. 
The  awarding  of  contracts  under  these  terms  will  be  contingent  upon  cost  or  price  issues. 
Due  to  the  weights  assigned,  the  effects  of  prior  past  performance  are  almost  negligible, 
and  it  seems  appropriate  that  the  companies  would  develop  their  pricing  strategies  to 
revolve  around  cost  or  price  factors. 

3.  Survey  Conclusions 

The  bulk  of  the  research  presented  here  supports  the  hypothesis.  It  appears  that 
companies  will  modify  their  pricing  strategies  and  accept  more  risk  when  past 
performance  is  rated  significantly  higher  than  cost  or  price,  and  that  past  performance 
will  have  little  or  no  effect  on  pricing  strategies  when  past  performance  is  rated 
significantly  less  than  cost  or  price. 
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It’s  interesting  to  note  that  at  least  one  of  the  assumptions  that  the  hypothesis  was 
based  upon  needs  to  be  modified.  While  the  researcher  expected  to  see  that  pricing  would 
be  higher  in  those  cases  when  companies  believed  that  they  held  the  edge  in  past 
performance,  the  research  indicates  that  these  companies  may  actually  employ  a  pricing 
strategy  that  results  in  prices  that  are  below  their  competitors. 

The  research  highlights  an  interesting  paradox.  The  Federal  acquisition  workforce 
has  adopted  the  use  of  past  performance  in  source  selections  because  business  leaders 
have  convinced  policy  makers  that  the  use  of  past  performance  information  is  one  of  their 
“best  practices.”  But  while  corporate  America  uses  past  performance  in  their  dealings 
with  suppliers,  these  same  companies  are  suspicious  when  Federal  acquisition 
professionals  use  this  same  best  practice. 

Because  companies  are  unsure  of  how  their  past  performance  will  be  rated  against 
their  competitors,  there  are  numerous  pricing  strategies.  The  implications  of  this  paradox 
are  displayed  in  the  following  example.  A  contracting  officer  receives  three  responses  to 
a  request  for  proposal  (RFP)  that  stated  that  the  contract  would  be  awarded  without 
discussions,  and  that  past  performance  would  be  weighted  significantly  higher  than  cost 
or  price.  The  independent  Government  cost  estimate  for  the  contract  is  $1,500,000. 

A  B  C 

$1,500,000  $1,350,00  $1,250,000 

Initially  Company  “C”  would  appear  to  be  the  best  choice  (lowest  price),  but 
through  detailed  analysis  of  the  proposals,  and  diligent  verification  of  the  resumes 
submitted  for  the  determination  of  past  performance,  the  contracting  officer  would  find 
that  Company  “A”  has  average  past  performance  and  a  standard  pricing  strategy  (a 
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forward  pricing  rate  agreement).  Company  “C”  has  average  past  performance,  and  its 
pricing  is  based  upon  a  critical  need  for  the  work.  Lastly,  Company  “B”  has  above 
average  past  performance,  and  in  order  to  win  the  contract  award  its  pricing  is  based  upon 
a  cost  challenge.  Clearly  Company  “B’s”  proposal  would  pose  the  least  amount  of  risk  to 
the  Government  and  its  past  performance/price  tradeoff  would  be  the  most  compelling 
(the  best  value). 

If  this  example  were  to  have  taken  place  in  a  highly  competitive  market,  the  use 
of  best  practices  would  be  critical  in  presenting  a  level  playing  field  to  the  offerors.  The 
solicitation  would  have  to  communicate  effectively  to  each  offeror.  The  determination  of 
relevant  PPI  to  be  considered  and  the  verification  of  that  PPI  would  be  critical  to 
identifying  trends.  PPI  evaluators  would  need  to  be  trained  in  the  evaluation  of  past 
performance.  The  documentation  of  the  tradeoff  between  past  performance  and  price 
would  be  critical  in  debriefing  the  unsuccessful  offerors  as  to  why  a  price  premium  was 
either  paid  or  not  paid.  To  forestall  protests  to  the  Comptroller  General,  each  debriefing 
would  have  to  provide  each  offeror  with  the  sense  that  its  proposal  was  evaluated  in  a  fair 
and  reasonable  manner. 

D.  SUMMARY 

In  this  chapter  the  researcher  reviewed  five  best  practices  guides,  he  identified 
eleven  key  best  practices  associated  with  past  performance  information,  and  he  reported 
on  a  survey  of  industry.  The  next  chapter  will  look  at  how  the  Comptroller  General  has 
dealt  with  recent  past  performance  protests. 
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IV.  PAST  PERFORMANCE  PROTESTS 


A.  INTRODUCTION 

This  chapter  begins  with  an  overview  of  the  GAO  Comptroller  General  protest 
process.  It  then  reports  on  an  analysis  of  all  protests  from  the  last  four  fiscal  years  to 
identify  overall  trends.  PPI  protests  are  then  compared  to  the  overall  population  of 
protests  to  determine  if  the  percentage  of  PPI  protests  are  increasing  or  decreasing. 
Following  the  analysis  of  the  general  trends,  the  sustained  PPI  protests  are  analyzed  to 
determine  what  specific  trends  have  developed.  After  the  general  and  specific  trend 
analysis,  all  one  hundred  fifty-one  PPI  protests  are  categorized,  and  fourteen  sustained 
and  denied  protests  are  reviewed  to  highlight  the  case  principles  that  the  Comptroller 
General  applies  when  issues  of  past  performance  are  involved. 

B.  THE  PROTEST  PROCESS 

The  following  is  the  researcher’s  interpretation  of  the  General  Accounting  Office, 
Administrative  Practice  and  Procedure,  Bid  Protest  Regulations,  Government  Contracts, 
4  CFR  Part  21,  effective  date  August  8, 1996.  [Ref.  17] 

The  bid  protest  process  begins  with  an  interested  party  filing  a  written  protest 
with  the  General  Accounting  Office  no  later  than  10  days  after  the  basis  for  the  protest  is 
known  or  should  have  been  known,  with  the  exception  of  protests  challenging  a 
procurement  conducted  on  the  basis  of  competitive  proposals  under  which  a  debriefing  is 
requested,  and  when  requested,  is  required.  In  such  cases,  the  initial  protest  shall  not  be 
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filed  before  the  debriefing  date  offered  to  the  protester,  but  shall  be  filed  not  later  than  10 
days  after  the  date  on  which  the  debriefing  is  held. 

Unless  the  protest  is  dismissed  due  to  procedural  or  substantive  defects  (if  the 
protest  passes  this  step  it  is  referred  to  as  a  merit  protest),  the  contracting  agency  must 
submit  an  agency  report  to  the  GAO  responding  to  the  protest,  and  a  copy  of  that  report 
must  be  provided  to  the  protester.  The  contracting  agency  may  also  be  required  to 
withhold  award  and  to  suspend  contract  performance  for  up  to  90  days  (90-day  stay),  to 
allow  the  Comptroller  General’s  office  to  complete  its  inquiry  into  the  protest  allegations. 
The  Competition  in  Contracting  Act  of  1984  authorizes  contracting  agencies  to  override 
the  stay  if  the  performance  of  the  contract  is  in  the  best  interest  of  the  Government  or  if 
there  are  justifiably  “urgent  and  compelling  circumstances.” 

During  the  protest  process,  the  Comptroller  General  may  schedule  informal 
meetings  or  conferences  to  resolve  procedural  matters  and  to  obtain  information 
pertaining  to  the  disposition  of  the  protest.  A  hearing  may  also  be  conducted  to  resolve 
factual  and  legal  issues  raised  during  the  protest  process.  Prior  to  the  hearing  a  pre- 
hearing  conference  will  be  conducted  to: 

•  Decide  what  issues  are  to  be  considered  during  the  hearing 

•  Identify  witnesses  who  will  testify 

•  Settle  any  outstanding  procedural  issues 

Within  100  days  after  the  hearing,  the  Comptroller  General  will  issue  a  decision. 
Protests  may  be  denied  or  sustained.  In  the  case  of  sustained  protests,  if  the  Comptroller 
General  determines  that  a  source  selection  was  not  conducted  in  accordance  with  current 
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statute  or  regulation,  he  shall  recommend  that  the  contracting  agency  implement  any 
combination  of  the  following  remedies: 

•  Terminate  the  contract 

•  Recompete  the  contract 

•  Issue  a  new  solicitation 

•  Award  the  contract  consistent  with  statute  or  regulation 

•  Pay  the  protestor’s  costs  of  filing  and  pursuing  the  protest,  to  include 
attorneys’  fees,  consultant  and  expert  witness  fees,  and  bid  and  proposal 
preparation  costs 

Unless  the  decision  contains  protected  information,  a  copy  will  be  provided  to  the 
protestor,  the  head  of  the  contracting  activity  involved,  the  senior  procurement  executive 
of  the  Federal  agency  involved,  and  a  copy  shall  be  made  available  to  the  public. 

C.  TREND  ANALYSIS 

During  the  collection  phase  of  information  to  analyze  for  trends,  Mr.  Jerold 
Cohen,  Special  Assistant  to  the  General  Counsel  (GAO),  informed  the  researcher  that 
while  the  GAO  did  keep  overall  statistics,  the  agency  did  not  keep  statistics  on  past 
performance  protests.  [Ref.  35]  Mr.  Cohen  advised  the  researcher  to  read  through  the 
Comptroller  General’s  decisions  available  through  the  GAO’s  database,  review  John 
Cibinic  and  Ralph  Nash’s  Formation  of  Government  Contracts,  3rd  ed.  [Ref.  8]  and  to 
contact  personnel  such  as  Mr.  Bob  Antonio,  Webmaster  for  “Where  in  Federal 
Contracting”.  [Ref.  50]  Table  3.1  was  developed  by  the  researcher  from  information 
obtained  from  the  Comptroller  General’s  Office,  Mr.  Antonio,  and  a  comprehensive 
review  of  the  protests  involving  past  performance. 
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GAO's  Bid  Protest  Statistics  ] 

Fiscal  Years 

1997 

1998 

1999 

2000  (Oct-Jun) 

Merit  Protests 
(Sustained  &  Denied) 

501 

406 

347 

161 

Protests  Sustained 

61 

63 

74 

44 

Sustainment  Rate  (%) 

12% 

16% 

21% 

27% 

Past  Performance  Statistics 

Fiscal  Years 

1997 

1998 

1999 

2000  (Oct-Jun) 

Past  Performance 

Protests  (Sustained  & 
Denied) 

40 

43 

62 

46 

Protests  Sustained 

6 

13 

15 

13 

Sustainment  Rate  (%) 

15% 

30% 

24% 

28% 

Comparison  C 

If  GAO  Statistics  &  Past  Performance  Statistics 

Fiscal  Years 

1997 

1998 

1999 

2000  (Oct-Jun) 

Past  Performance  Protests 
as  a  Percentage  of  Merit 
Protests 

8% 

11% 

18% 

29% 

Past  Performance  Protests 
as  a  Percentage  of 
Sustained  Protests 

10% 

21% 

20% 

30% 

Table  4. 1 .  Past  Performance  Protests. 

Analysis  of  the  data  shows  a  number  of  trends  that  have  developed  since  the 

introduction  of  the  requirement  to  utilize  past  performance  information  in  the  Federal 

acquisition  process.  The  first  major  trend  is  that  while  the  number  of  merit  protests  has 

been  declining  over  the  last  four  years,  the  number  of  past  performance  related  protests 

has  been  increasing.  In  1997,  past  performance  protests  constituted  only  8%  of  all  merit 

protests,  and  as  of  June  2000  that  percentage  has  grown  to  29%.  This  upward  trend  has 
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also  taken  place  with  the  percentage  of  past  performance  protests  as  a  percentage  of 
sustained  protest.  In  1997,  sustained  past  performance  protests  accounted  for  only  10%  of 
the  overall  sustained  protests,  and  by  June  of  2000  that  percentage  has  risen  to  30%. 

Both  of  these  trends  are  consistent  with  the  research  presented  earlier  in  this 
thesis,  that  offerors  are  suspicious  and  uncomfortable  with  the  use  of  PPI  by  Federal 
acquisition  professionals.  It  is  understandable  that  as  the  dollar  thresholds  for  the 
collection  and  use  of  PPI  have  decreased  from  $1,000,000  in  1997,  to  $100,000  in 
January  1999,  businesses  would  respond  to  the  increased  use  of  PPI  in  source  selections 
by  questioning  its  use. 

In  Chapter  II,  it  was  pointed  out  that  the  Federal  acquisition  workforce  has  had 
some  difficulty  implementing  various  acquisition  reform  initiatives  that  had  been  levied 
upon  them  in  the  last  five  years.  The  struggle  to  incorporate  the  collection  and  evaluation 
of  PPI  into  the  day-to-day  operations  of  Federal  procurement  agencies  is  reflected  in  the 
increasing  number  of  sustained  past  performance  related  protests. 

During  an  interview,  Mr.  Stan  Soloway,  the  Deputy  Under  Secretary  of  Defense 
(Acquisition  Reform)  told  the  researcher  that  he  was  not  surprised  that  the  number  of  past 
performance  related  protests  had  increased  and  that  the  number  of  sustained  past 
performance  protests  had  also  increased.  He  explained  that  DoD  officials  should  have 
seen  these  trends  coming  because: 

Businesses  want  to  be  treated  fairly.  They  want  a  level  playing  field,  and 
they  can’t  be  guaranteed  one  with  our  current  collection  and  tracking 
systems.  With  the  increased  emphasis  on  the  collection  and  use  of  past 
performance  information,  it  is  understandable  that  businesses  would 
challenge  more  of  our  evaluations  and  best  value  determinations. 
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He  went  on  to  further  explain  that: 


At  one  time  there  was  talk  of  a  DoD-wide  Contractor  Performance  and 
Assessment  Reporting  System  (CPARS).  Recent  developments  have 
changed  that  goal,  and  now  each  Service  is  going  to  be  responsible  for  its 
own  system.  In  the  absence  of  a  good  collection  and  tracking  system  for 
past  performance,  there  is  always  the  possibility  of  mistakes  in  the 
collection  and  evaluation  of  past  performance.  These  mistakes  translate 
into  more  past  performance  protests  that  have  substance. 

Mr.  Soloway  concluded  the  discussion  by  stating  that: 


Because  of  the  benefits  associated  with  the  use  of  past  performance 
information  in  source  selections,  we  would  never  want  to  go  back  to  the 
old  ways  of  looking  at  cost  or  price  only,  [Low  price  technically 
acceptable  (LPTA)]. 

Improving  our  processes  for  the  collection  and  use  of  past  performance 
information  throughout  the  Federal  acquisition  process  is  one  of  the  keys 
for  ensuring  that  businesses  that  deal  with  us  feel  that  they  are  being 
treated  fairly.  Their  continued  interest  in  doing  business  with  the  Federal 
Government  is  essential  to  continued  competition  in  Federal  contracting 
[Ref.  18] 

D.  SUSTAINED  PROTEST  BREAKOUT 


Figure  4.1  was  developed  from  an  analysis  of  the  forty-one  sustained  past 
performance  protests  from  Fiscal  Year  1998  through  the  first  three  quarters  of  Fiscal 
Year  2000.  It  highlights  the  fact  that  in  the  majority  of  the  sustained  protests,  Federal 
acquisition  professionals  failed  to  consistently  apply  their  own  evaluation  criteria.  In 
thirty-three  of  the  sustained  protests,  the  evaluation  of  past  performance  information  was 
found  not  to  be  consistent  with  the  evaluation  criteria  contained  in  the  request  for 
proposals  (RFP),  or  the  source  selection  was  found  to  be  arbitrary  and  capricious  with 
little  or  no  regard  for  PPI.  It  is  evident  that  in  these  protests,  Government  acquisition 
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professionals  failed  to  adhere  to  the  current  regulatory  requirements  and  the  published 
best  practices. 


SUSTAINED  PROTEST  BREAKOUT 


10 


12 


14 


Figure  4.1.  Analysis  of  Sustained  Past  Performance  Protests. 

While  there  are  lessons  to  be  learned  from  the  sustained  protests,  there  are  more 

lessons  to  be  learned  from  the  complete  population  of  past  performance  related  protests. 

Because  the  Comptroller  General  will  only  rule  on  a  protest  after  it  has  been  determined 

that  the  protest  has  merit,  it  follows  that  there  are  lessons  to  be  learned  from  both 

sustained  and  denied  protests.  The  population  of  one  hundred  fifty-one  past  performance 

related  protests  from  Fiscal  Year  1998  through  June  2000  (Appendix  C)  can  be 

categorized  into  seven  general  categories: 
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•  Evaluation  Plans 

•  Evaluation  Criteria  or  Factors 

•  Past  Performance 

•  Best  Value  Evaluations 

•  Evaluation  of  Offers 

•  Bias  of  Government  Officials 

•  Proposal  Evaluation— Documentation 

The  categories  facilitate  the  identification  of  the  key  case  principles  that  can  be 
identified  from  both  the  sustained  and  the  denied  past  performance  protests.  Knowledge 
of  and  an  understanding  of  these  case  principles  will  enable  Federal  acquisition 
professionals  to  more  effectively  utilize  past  performance  information  in  source 

selections. 

E.  CASE  PRINCIPLES 

In  identifying  the  case  principles  that  apply  to  each  category,  representative 
protests  will  be  used  to  illustrate  the  principles.  Each  protest  case  will  be  identified,  the 
protestor’s  position  will  be  briefly  reviewed,  the  process  the  Comptroller  General  (Comp. 
Gen.)  used  to  review  the  case  will  be  briefly  noted,  and  then  each  protest  ruling  will  be 
tied  to  the  key  case  principle  that  the  Comp.  Gen.  relied  upon  in  either  sustaining  or 
denying  the  protest.  For  later  reference,  a  listing  of  the  principles  derived  from  these 
cases  is  presented  in  Appendix  D. 
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1. 


Evaluation  Plans 


a.  Matter  of:  General  Security  Services  Corporation,  B-280388;  B- 
280388.2,  September  25, 1998 

In  this  case,  General  Security  Services  Corporation  (protestor)  argued  that 
there  was  a  lack  of  documentation  in  the  evaluation  record  that  justified  scoring  changes 
related  to  its  proposal  under  the  assessment  of  the  past  performance  sub-factor.  In 
particular,  the  record  failed  to  include  separate  evaluation  forms  to  explain  the  changed 
scoring,  as  required  by  internal  agency  instructions,  entitled  "Instructions  for  Technical 
Evaluation  Factors."  In  support  of  its  position,  the  protester  cites  one  evaluator's  scoring 
sheet,  which  showed  "what  appeared  to  be  a  scoring  change  from  four  points  to  zero 
points  on  the  past  performance  sub-factor  which  was  worth  5  points." 

After  reviewing  the  records  the  Comp.  Gen.  denied  the  protest.  The  denial 
was  based  on  the  principle  that: 

CGP  la.  Alleged  deficiencies  in  the  application  of  an  agency’s  evaluation 
plan  does  not  provide  a  basis  for  questioning  the  validity  of  the  award 
selection;  these  plans  are  internal  agency  instructions  and  as  such  do  not 
give  outside  parties  any  rights.  Consequently,  the  fact  that  an  agency  may 
not  have  followed  its  internal  evaluation  instructions  is  not  a  valid  basis 
for  protest.  (Ref.  24) 

b.  Matter  of:  Enmax  Corporation,  B-281965,  May  12, 1999 

In  this  case  Enmax  Corporation  (protestor)  argued  that  the  Air  Force’s 

(agency)  evaluation  of  the  awardee’s  proposal  was  improper  in  two  areas-the  conclusion 
that  the  awardee’s  proposal  was  technically  acceptable,  and  the  decision  that  the 
awardee’s  proposal  presented  low  performance  risk.  In  considering  a  protest  challenging 
an  agency's  evaluation  of  proposals,  the  Comp.  Gen.  will  examine  the  record  to 
determine  whether  the  agency's  judgment  was  reasonable  and  consistent  with  stated 
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evaluation  criteria  and  applicable  statutes  and  regulations.  In  this  case,  the  Comp.  Gen. 
found  that  the  agency’s  evaluators  had  failed  to  evaluate  the  awardee’s  proposal  in 
accordance  with  the  evaluation  criteria  contained  in  the  solicitation.  The  protest  was 
sustained,  and  the  sustainment  was  based  on  the  principle  that: 

CGP  lb.  As  a  preliminary  matter,  it  is  the  solicitation,  not  the  internal 
evaluation  plan  that  is  the  touchstone  for  whether  offerors  have  been 
treated  fairly  in  an  evaluation.  Requirements  stated  in  evaluation  and 
source  selection  plans  are  not  disclosed  to  offerors,  and  thus,  do  not  give 
outside  parties  any  rights.  [Ref.  23] 

2.  Evaluation  Criteria/Factors 

a.  Matter  of:  Borders  Consulting,  Inc.,  B-281606,  March  10, 1999 
In  this  case  the  protestor’s  (Borders  Consulting,  Inc.)  principal  argument 

was  that  the  solicitation's  evaluation  criteria  were  improper  and  that  certain  of  the 

agency's  technical  evaluators  should  have  been  excluded  from  participation  in  the 

acquisition.  The  protestor  took  specific  issue  with  two  criteria-experience  and 

knowledge  of  secure  electronic  commerce  development,  and  experience  and  knowledge 

of  web-based  security— asserting  that  they  do  not  reflect  the  statement  of  work.  The 

protestor  also  asserted  more  generally  that  the  evaluation  criteria  were  designed  to  favor 

the  incumbent  contractor.  After  reviewing  the  records,  the  Comp.  Gen.  ruled  that  the 

protest  that  the  evaluation  criteria  were  designed  to  "steer"  the  award  to  a  particular  firm 

was  denied  where  the  criteria  directly  related  to  statement  of  work.  The  denial  was  based 

on  the  principle  that: 

CGP  2.  Agencies  enjoy  broad  discretion  in  selecting  evaluation  criteria 
and  the  Comptroller  General  will  not  object  to  a  solicitation's  evaluation 
scheme  so  long  as  it  reasonably  relates  to  the  agency's  needs.  The  fact  that 
a  solicitation's  requirements  or  evaluation  criteria  may  favor  the  agency's 
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actual  needs,  and  the  advantage  enjoyed  by  a  particular  firm  is  not  the 
result  of  improper  Government  action.  [Ref.  20] 

3.  Past  Performance 

a.  Matter  of:  OMV Medical,  Inc.;  Saratoga  Medical  Center,  Inc.,  B- 
281606,  February  3, 1999 

In  this  case  both  protesters  (OMV  Medical,  Inc.  and  Saratoga  Medical 
Center,  Inc.)  principally  assert  that  the  agency  failed  to  adhere  to  the  RFP's  announced 
evaluation  standard,  relaxed  the  RFP's  adequate  compensation  requirements,  failed  to 
meaningfully  evaluate  price  realism  and  misled  them  into  failing  to  reduce  their  proposed 
professional  compensation.  The  protesters  also  contended  that  the  Air  Force  (agency) 
engaged  in  prejudicially  unequal  discussions  with  certain  offerors. 

The  protestors  also  contended  that  the  agency  arbitrarily  neutralized  past 
performance  as  an  evaluation  discriminator  by  according  all  offerors  "low  risk" 
performance  ratings  regardless  of  experience,  which  was  allegedly  prejudicial  to  the 
incumbent.  The  Comp.  Gen.  found  that  here,  the  main  purpose  of  the  past  performance 
evaluation  was  for  the  agency  to  identify  and  review  relevant  present  and  past 
performance  in  order  to  make  an  overall  risk  assessment  of  the  offeror's  ability  to 
perform  the  requirement.  In  order  to  do  so,  the  agency  sent  questionnaires  to  a  minimum 
of  two  references  provided  by  each  offeror.  Based  on  the  responses  received,  the  agency 
concluded  that  all  offerors  were  capable  of  performing  and,  thus,  all  received  a  low 
performance  risk  rating.  Although  the  protestor  challenged  the  relevance  of  the  references 
submitted  by  some  offerors,  the  Comp.  Gen.  found  nothing  unreasonable  in  the  agency’s 
approach  to  investigating  the  past  performance  history  of  the  offerors  and,  based  on  that 
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investigation,  in  concluding  that  all  offerors  presented  a  low  risk  of  nonperformance. 
Therefore,  the  protest  was  denied  and  the  denial  was  based  upon  the  principle  that: 

CGP  3a.  Where  a  solicitation  requires  the  evaluation  of  offerors'  past 
performance,  an  agency  has  the  discretion  to  determine  the  scope  of  the 
offerors'  performance  histories  to  be  considered  provided  all  proposals  are 
evaluated  on  the  same  basis  and  consistent  with  the  solicitation 
requirements.  [Ref.  31] 

b.  Matter  of:  D.F.  Fire  Fighter  Catering,  B-280767.4,  September 
10, 1999 

In  this  case,  the  D.  F.  Fire  Fighter  Catering  (protestor)  argued  that  the  U.S. 
Forest  Service  (agency)  failed  to  comply  with  FAR  42.1503  (b)  and  (e)  in  its  evaluation 
of  the  protestor’s  past  performance.  Specifically,  the  protestor  argued  that  the  agency  did 
not  review  certain  disagreements  regarding  the  protestor’s  past  performance  at  a  level 
above  the  contracting  officer,  as  required  by  FAR  42.1503  (b),  and  that  the  agency  was 
precluded  by  FAR  42.1503  (e)  from  considering  adverse  PPI  from  a  prior  contract 
because  that  information  was  more  than  three  years  old. 

After  reviewing  the  records  of  the  case,  the  Comp.  Gen.  found  that 
contrary  to  the  protestor’s  assertions,  both  the  protestor’s  PPI  and  the  contracting 
officer’s  comments  concerning  the  protestor’s  PPI  were  reviewed  by  the  Head  of  the 
Contracting  Activity  and  the  Office  of  General  Counsel.  The  Comp.  Gen.  also  ruled  that 
the  agency  complied  with  FAR  42.1503(e),  which  provides  that  "past  performance 
information  shall  not  be  retained  to  provide  source  selection  information  for  longer  than  3 
years  after  completion  of  contract  performance,"  even  though  it  considered  all  of  the 
protester's  5-year  past  performance  history  under  the  predecessor  contract,  because  the 
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past  performance  evaluation  took  place  within  3  years  of  completion  of  that  contract's 
performance.  The  protest  was  denied,  and  the  denial  was  based  upon  the  principle  that. 

CGP  3b.  A  contractor’s  past  performance  may  be  considered  for  up  to 
three  years  after  the  completion  of  contract  performance  as  a  whole,  rather 
than  for  only  three  years  after  each  incident  of  performance  under  the 
contract.  [Ref.  22] 

c.  Matter  of:  Kellie  W.  Tipton  Construction  Company,  B-281331.3, 
March  22, 1999 

In  this  case,  the  Kellie  W.  Tipton  Construction  Company  (protestor) 
protested  the  termination  for  convenience  of  its  contract  for  the  installation  and 
replacement  of  water  and  sewer  lines  at  Fort  Campbell,  Kentucky  and  the  Army's 
(agency)  decision  to  award  a  contract  for  these  services  to  another  offeror. 

The  Comp.  Gen.  ruled  that  the  allegation  that  the  agency  misevaluated 
proposals  with  respect  to  past  performance  was  denied  where  the  record  showed  that  the 
agency  evaluation  was  reasonable  and  did  not  conflict  with  the  solicitation  evaluation 
criteria.  While  the  solicitation  called  for  offerors  to  submit  at  least  ten  references,  and  the 
awardee  did  so,  the  solicitation  did  not  state  that  all  ten  references  would  be  evaluated, 
and  the  agency's  decision  to  evaluate  only  five  references  for  all  offerors  did  not 
prejudice  the  protester  and  was  otherwise  unobjectionable. 

The  agency  reported  that  while  it  requested  and  would  have  preferred  to 

obtain  completed  questionnaires  from  ten  of  the  contractor's  job  references,  not  all 

offerors  submitted  ten  references  and  the  agency  determined  that  five  past  performance 

records  were  sufficient  to  evaluate  an  offeror.  In  evaluating  the  protestor’s  past 

performance  the  agency  reviewed  three  government  projects  ratings  and  two  customer 

performance  surveys  and  concluded  that  there  was  a  high  probability  that  the  protestor 
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would  perform  well.  Likewise,  the  agency  reviewed  three  government  projects  and  two 
customer  performance  surveys  in  its  review  of  the  awardee’s  past  performance.  The 
protestor  and  the  awardee  were  rated  as  being  equal  under  past  performance. 

Notwithstanding  the  protestor’s  assertions  to  the  contrary,  the  Comp.  Gen. 
found  nothing  unreasonable  in  the  agency's  manner  of  assessing  the  awardee’s  and  the 
protestor’s  past  performance.  While  the  RFP  requested  a  minimum  of  ten  references,  it 
did  not  specify  the  number  of  references  that  the  agency  would  contact  for  purposes  of 
evaluation.  The  protest  was  denied  and  the  ruling  was  based  upon  the  principle  that: 

CGP  3c.  There  is  no  requirement  that  an  agency  contact  all  of  an  offeror's 
references.  [Ref.  27] 

d.  Matter  of:  Consolidated  Engineering  Services  Inc.,  B-279565.2; 

B-2 79565.3,  June  26, 1998 

Consolidated  Engineering  Services,  Inc.  (protestor)  protested  the 
Department  of  Housing  &  Urban  Development's  (agency)  award  of  a  contract  to  Halifax 
Technical  Services  (awardee)  for  commercial  facilities  management  services  with  respect 
to  the  HUD  headquarters  building  in  Washington,  D.C.  CESI  challenged  the  evaluation 
results. 

The  protestor  argued  that  the  agency  improperly  downgraded  its  proposal 
relative  to  the  awardee's  based  on  the  awardee's  more  detailed  description  of  the  proposed 
elevator  maintenance  subcontractor’s  experience;  since  the  protester  and  the  awardee 
proposed  using  the  same  subcontractor,  they  should  have  received  the  same  score  for  the 
subcontractor's  experience. 
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The  protestor  also  challenged  the  evaluation  of  the  subfactor  for  proposed 
elevator  maintenance  organization,  under  the  experience  and  qualifications  factor  £^he 
awardee's  proposal  received  150  and  the  protestor's  received  142  of  the  available  160 
points  under  the  factor).  Although  both  offerors  proposed  the  incumbent  HUD  elevator 
maintenance  subcontractor,  3  of  15  available  subfactor  points  were  deducted  from  the 
protestor’s  score— while  the  awardee  received  all  15  points- for  failure  to  adequately 
address  the  results  achieved  (e.g.,  quality  of  service,  timeliness  of  performance  and  cost 
control),  by  the  subcontractor  under  prior  contracts.  The  protestor  disputes  the  agency's 
position  that  the  awardee’s  proposal  was  superior  in  addressing  the  results  achieved  by 
the  subcontractor,  and  argues  that,  in  any  case,  since  the  proposed  subcontractor  was  the 
incumbent,  the  evaluators  should  have  been  aware  of  its  performance  and  capabilities. 

The  Comp.  Gen.  ruled  that  even  if  the  agency  was  correct  that  the 
protestor’s  proposal  did  not  provide  as  much  information  as  the  awardee  s  regarding  the 
subcontractor’s  experience,  since  both  proposals  offered  the  same  subcontractor,  the 
evaluation  unreasonably  accorded  the  two  proposals  different  scores  in  this  area.  Once 
the  agency  became  aware  of  the  subcontractor’s  experience— whether  from  the  awardee’s 
proposal,  personal  knowledge,  or  otherwise— it  could  not  have  reasonably  assigned  the 
awardee’s  proposal  a  higher  score  than  the  protestor’s  based  on  that  experience. 
Accordingly,  it  appears  that  the  protestor’s  proposal  was  entitled  to  the  same  score  as  the 
awardee’s  under  this  subfactor,  for  an  increase  of  3  points.  The  protest  was  sustained,  and 
the  ruling  was  based  upon  the  principle  that: 

CGP  3d.  An  agency  may  not  ignore  prior  performance  information  of 
which  it  is  aware.  (Ref.  21) 


47 


4. 


Best  Value  Evaluations 


a.  Matter  of:  Marathon  Watch  Company  Limited,  B-281876;  El- 
28 1876.2,  April  22,  1999 

Marathon  Watch  Company  Limited  (protestor)  protested  the  issuance  of  a 
purchase  order  by  the  Defense  Logistics  Agency,  Defense  Supply  Center  Richmond 
(agency),  to  Stocker  &  Yale,  Inc.  (awardee).  The  protestor  challenged  the  adequacy  of 
the  best  value  determination,  which  resulted  in  the  agency’s  issuance  of  the  purchase 
order  to  S&Y. 

The  Comp.  Gen.  found  that  the  agency’s  selection  of  a  higher-priced 
vendor  with  excellent  performance  history  instead  of  a  lower-priced  vendor,  whose 
performance  reflected  delivery  delinquencies,  was  reasonable  and  consistent  with  the 
solicitation's  evaluation  scheme,  which  provided  that  if  the  vendor  with  the  best  past 
performance  history  did  not  offer  the  lowest  price,  the  agency  would  make  a  tradeoff  of 
price  and  past  performance  including  such  considerations  as  inventory  status  and 
historical  delivery  problems,  where  there  was  a  low  inventory  of  the  solicited  item  and 
the  higher-priced  vendor's  better  performance  record  indicated  a  lower  performance  risk. 

In  reviewing  the  records,  the  Comp.  Gen.  found  that  the  request  for 
quotations  (RFQ)  advised  that  the  best  value  determination  would  be  based  on  a 
comparative  assessment  of  prices  and  past  performance,  which  were  equally  weighted. 
The  past  performance  factor  considered  quality  performance  and  delivery  performance  to 
be  "of  equal  value."  If  the  vendor  with  the  best  past  performance  history  did  not  offer  the 
lowest  price,  the  agency  would  make  the  appropriate  tradeoff  of  price  for  past 
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performance,  and  listed  several  considerations  such  as  delivery  schedule/inventory  status 
and  historical  delivery/quality  problems  that  could  affect  the  tradeoff  determination. 

Based  upon  the  review  of  the  records,  the  protest  was  denied  and  the 
Comp.  Gen  based  the  decision  upon  the  principle  that: 

CPG  4a.  When  making  “Best  Value  Determinations”,  the  Contracting 
Officer’s  decisions  concerning  tradeoffs  between  past  performance  and 
cost/price  will  be  judged  to  be  reasonable  and  consistent,  provided  that  it 
can  be  shown  that  the  tradeoffs  are  in  the  best  interest  of  the  government 
and  that  they  are  consistent  with  the  solicitation’s  evaluation  scheme.  [Ref. 

29] 

b.  Matter  of:  Beneco  Enterprises,  Inc.,  B-283154,  October  13, 1999 

Here  Beneco  Enterprises,  Inc.  (protestor)  protested  the  awarding  of  three 

contracts  to  [OfferorA],  Straub/Pacific  LLC,  and  [Offeror  B],  by  the  U.S.  Army  Corps  of 
Engineers  (agency),  for  construction  services  at  installations  under  the  jurisdiction  of  the 
Corps's  Los  Angeles  District  and  the  Bureau  of  Prisons'  Western  Region.  Beneco 
protested  that  the  Corps  misevaluated  price  and  did  not  treat  it  as  significantly  less 
important  than  technical  concerns  in  the  price/technical  tradeoff,  as  required  by  the  RFP 
evaluation  scheme. 

The  Comp.  Gen.  found  that  the  contemporaneous  records  did  not  explain 
why  the  agency  rejected  the  protestor’s  significantly  superior  proposal  to  save  an 
evaluated  $534,433  represented  by  [Offeror  B's]  proposal-a  proposal  which  earned  less 
than  two-thirds  of  the  available  evaluation  points,  garnered  only  "satisfactory"  ratings 
under  evaluation  factors  representing  850  evaluation  points,  and  was  the  source  of  grave 
concern"  regarding  its  past  performance/quality.  The  contemporaneous  record  simply 
reflected  that  the  agency  was  unwilling  to  pay  a  27-percent  price  premium  to  obtain  a 
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proposal  with  a  36-percent  higher  technical  score,  without  any  discussion  of  the 
proposal’s  relative  strengths  and  weaknesses.  Furthermore,  there  was  no  indication  in  the 
contemporaneous  record  that  the  agency,  in  reaching  this  conclusion,  accounted  for  the 
fact  that  technical  factors  were  significantly  more  important  than  price  factors  under  the 
stated  evaluation  scheme. 


While  the  Comp.  Gen.  did  agree  that  after  evaluations,  the  selection 
authority  might  make  price/technical  tradeoffs;  the  extent  to  which  one  may  be  sacrificed 
for  the  other  is  governed  by  the  test  of  rationality  and  consistency  with  the  established 
evaluation  factors.  In  this  case  the  protest  was  sustained  and  the  Comp.  Gen.  based  the 
ruling  in  part  upon  the  principle  that: 

CGP  4b.  When  price  is  secondary  to  non-cost/price  factors,  an  agency 
may  not  select  a  lower-priced,  lower-scored  proposal,  unless  it  determines 
that  the  premium  involved  in  selecting  a  higher-rated,  higher-priced 
proposal  is  not  justified.  However,  when  selection  officials  reasonably 
regard  proposals  as  being  essentially  equal,  price  can  become  the 
determining  factor  in  making  an  award  notwithstanding  that  the  evaluation 
criteria  assigned  price  less  importance  than  non-cost/price  factors.  [Ref. 

19] 


c.  Matter  of:  Schaeffer  Eye  Center,  B-284268,  March  20,  2000 
Schaeffer  Eye  Center  (protestor)  protested  the  award  of  a  contract  to 

Classic  Optical  Laboratories,  Inc.  (awardee)  by  the  Department  of  Veterans  Affairs 

(agency)  for  prescription  eyeglasses  for  eligible  veterans  at  VA  medical  centers  in 

Alabama,  Georgia,  and  South  Carolina.  The  protestor  contended  that  the  award  on  the 

basis  of  the  awardee’s  lower-priced,  lower-rated  technical  proposal  was  not  in  accordance 

with  the  RFP's  stated  evaluation  scheme,  which  provided  that  technical  factors  were  more 

important  than  price. 
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The  Comptroller  General’s  review  of  the  contemporaneous  narrative 
evaluation  findings  for  each  of  the  offers  under  each  of  the  evaluation  factors  did  not 
provide  any  basis  to  question  the  agency's  determination  of  technical  equality.  For 
instance,  both  offerors'  proposals  were  credited  for  providing  solid  documentation  of 
their  proposed  approaches,  extensive  experience,  and  personnel  qualifications;  the 
awardee's  proposal,  in  fact,  was  rated  technically  outstanding  in  more  evaluation  areas 
than  the  protestor’s  proposal.  The  awardee’s  proposal  was  noted  for  exceeding  certain 
important  RFP  requirements,  surpassing  the  extent  by  which  the  protestor’s  proposal 
exceeded  the  same  requirements. 

In  the  Comp.  Gen.  judgment,  when  a  selection  official  determines  that 
proposals  are  technically  equal,  it  means  that  overall  there  is  no  meaningful  difference  in 
what  the  proposals  have  to  offer.  It  does  not  mean  that  the  proposals  are  identical  in  every 
respect;  one  may  be  superior  to  the  other  in  a  variety  of  areas.  In  this  case,  the  protestor 
did  not  argue  with  any  specificity  as  to  what  aspects  of  its  proposal  render  it  overall 
technically  superior  to  the  awardee’s  proposal.  Rather,  the  protestor  relied  only  on  the 
slightly  higher  technical  score  its  proposal  received  to  assert  its  claimed  superiority, 
which,  as  discussed  above,  is  alone  insufficient  to  question  the  agency's  technically  equal 
determination. 

Consequently  the  protest  was  denied,  since  the  Comp.  Gen.  review  of  the 
records  provided  no  support  for  the  protester’s  claimed  technical  superiority.  The  Comp. 
Gen.  found  no  basis  to  object  to  the  reasonableness  of  the  agency's  determination  that  the 
proposals  were  essentially  technically  equal;  and  that  the  awardee’s  lower  evaluated  price 
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consequently  served  as  a  proper  determinative  factor  for  award.  The  ruling  was  based  in 
part  upon  the  principle  that: 

CGP  4c.  In  a  negotiated  procurement  with  a  best  value  evaluation  plan, 
point  scores  are  useful  as  guides,  but  they  do  not  mandate  automatic 
selection  of  a  particular  proposal.  Whether  a  given  point  spread  between 
two  competing  proposals  indicates  a  significant  superiority  of  one 
proposal  over  another  depends  upon  the  facts  and  circumstances  of  each 
procurement  and  is  primarily  a  matter  within  the  discretion  of  the 
procuring  agency.  The  question  of  whether  a  difference  in  point  scores  is 
significant  is  for  determination  on  the  basis  of  what  that  difference  might 
mean  in  terms  of  contract  performance  and  what  it  would  cost  the 
government  to  take  advantage  of  it.  [Ref.  32] 

5.  Evaluation  of  Offers 

a .  Matter  oft  LB&B  Associates ,  B-281 706,  March  24, 1999 

LB&B  Associates,  Inc.  (protestor)  protested  the  awarding  of  a  contract  to 

Four  Seasons  Environmental,  Inc.  (awardee)  by  the  Department  of  Agriculture, 
Agricultural  Research  Service  (agency),  for  operations  and  maintenance  support  services 
for  the  agency’s  South  Atlantic  Area  facilities.  The  protester  contended  that  the  agency 
misevaluated  its  proposal  and  misled  it  during  discussions. 

The  protester's  argument  that  it  should  have  received  a  higher  past 
performance  score  than  the  awardee  under  a  solicitation  calling  for  work  in  a  "hot" 
laboratory,  i.e.,  a  laboratory  in  which  live  viruses  are  present,  since  it  has  experience 
working  in  a  "hot"  laboratory,  whereas  awardee  does  not,  is  denied,  where  the  records 
demonstrated  that  the  awardee  does  in  fact  have  experience  working  in  a  "hot" 
laboratory. 


To  the  extent  that  the  protester  argued  that  its  proposal  deserved  a  higher 
score  under  the  Demonstrated  Success  in  Laboratory  subfactor,  the  Comp.  Gen.  ruled  that 
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it  was  not  the  role  of  its  Office  to  make  independent  determinations  regarding  the  merits 
of  technical  proposals.  The  protest  was  denied  and  the  Comp.  Gen  based  the  decision  on 
the  principle  that: 

CGP  5.  The  evaluation  of  proposals  is  within  the  discretion  of  the 
procuring  agency  since  it  is  responsible  for  defining  its  needs  and  the  best 
method  of  accommodating  them,  and  must  bear  the  burden  arising  from  a 
defective  evaluation.  The  Comptroller  General’s  function  is  to  examine 
the  agency  evaluation  to  ensure  that  it  was  reasonable  and  consistent  with 
the  stated  evaluation  factors. 

6.  Bias  of  Government  Officials 

a.  Matter  of:  J.A.  Jones  Grupo  de  Servicio,  SA,  B-283234,  October 
25, 1999 

J.A.  Jones  Grupo  de  Servicio,  SA,  (protestor)  protested  the  awarding  of  a 
contract  to  InteijetServe  (awardee)  by  the  Naval  Regional  Contracting  Center  in  Naples, 
Italy  (agency),  for  air  terminal  services  at  Rota,  Spain.  The  protester  asserted  that  the 
agency  was  biased  in  favor  of  the  awardee,  that  the  evaluation  of  proposals  was 
unreasonable  and  inconsistent  with  the  solicitation  criteria,  and  that  the  selection  of  the 
awardee  was  unjustified. 

The  protester  contended  that  the  contracting  officer,  who  performed  the 
past  performance  evaluation,  drafted  and  essentially  controlled  the  written  form  of  the 
evaluators’  findings,  and  made  the  selection  decision,  which  the  contract  review  board 
approved,  was  biased  in  favor  of  the  awardee.  The  protestor  essentially  contended  that 
the  ’’outstanding"  ratings  received  by  the  awardee  were  irrational,  considering  the  merits 
of  the  two  proposals,  and  that  there  was  no  basis  to  pay  a  premium  for  the  awardee's 
proposal. 
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The  Comp.  Gen.  review  of  the  records  found  that  a  message  had  been  sent 
prior  to  the  submission  of  initial  offers,  reminding  the  incumbent  contractor  to  follow  the 
solicitation’s  instructions  for  submitting  information  for  evaluation,  rather  than  rely  upon 
the  agency's  familiarity  with  incumbent's  capabilities.  The  Comp.  Gen.  saw  nothing 
improper  in  the  exchange,  which  occurred  prior  to  submission  of  initial  offers;  rather,  the 
advice  provided  by  the  contracting  officer  appeared  to  have  been  no  more  than  a 
conscientious  effort  to  enhance  competition  by  ensuring  that  a  significant  competitor,  the 
incumbent  contractor,  avoided  a  mistake  often  made  by  incumbents,  which  is  failing  to 
provide  information  needed  for  the  evaluation,  in  the  belief  that  the  agency  already  has 
that  information.  In  fact,  the  message  in  this  case,  refers  to  the  contracting  officer's  past 
experience  with  such  problems.  The  Comp.  Gen.  found  no  basis  to  object  to  the 
contracting  officer's  "gentle  advice,"  apparently  given  for  the  purpose  of  enhancing 
competition. 

The  Comp.  Gen.  review  of  the  written  records  found  that  the  solicitation 
provided  for  an  award  based  on  best  value,  considering  technical  acceptability,  evaluated 
on  a  pass/fail  basis,  price,  and  relative  capability.  Relative  capability  was  to  be 
determined  by  the  agency  on  the  basis  of  past  performance  and  an  understanding  of 
Government  requirements.  The  RFP  stated  that  the  agency  would  contact  each  offeror’s 
customers  for  information  on  past  performance,  defined  as  "a  measure  of  the  degree  to 
which  an  offeror  satisfied  its  customers  in  the  past."  It  directed  offerors  to  "describe  your 
company's  past  performance  on  directly  related  or  similar  Government  or  commercial 
contracts  and  subcontracts  that  are  similar  in  scope,  magnitude  and  complexity  to  that 
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required  by  the  RFP,"  including  current  contracts  and  those  completed  within  the  prior 
five  years. 


The  Comp.  Gen.  denied  the  protest  in  this  case  and  the  ruling  was  based 
upon  the  principle  that: 

CGP  6.  Because  government  officials  are  presumed  to  act  in  good  faith, 
we  do  not  attribute  unfair  or  prejudicial  motives  to  them  on  the  basis  of 
inference  or  supposition.  Where  a  protester  alleges  bias  on  the  part  of 
government  officials,  the  protester  must  provide  credible  evidence  clearly 
demonstrating  a  bias  against  the  protester  or  for  the  awardee  and  showing 
that  the  agency's  bias  translated  into  action  that  unfairly  affected  the 
protester's  competitive  position.  [Ref.  28] 

7.  Proposal  Evaluation — Documentation 

a.  Matter  of:  J&J  Maintenance  Inc.,  B-284708.2;  B-284708.3, 
June  5,  2000 

J&J  Maintenance,  Inc.  (protestor)  protested  the  Army’s  (agency)  award  of 
a  contract  to  Day  &  Zimmermann  Services  (awardee).  The  protester  contends  that  the 
decision  to  award  to  the  awardee  on  the  basis  of  its  higher-priced  proposal  was  flawed 
because  the  agency  unreasonably  downgraded  the  protester's  proposal  and  evaluated 
proposals  unequally,  giving  the  awardee’s  proposal  higher  ratings  in  a  number  of  areas 
even  though  the  protestor’s  proposal  contained  similar  or  better  features. 

The  Comp.  Gen.  review  of  the  records  found  that  minimal  documentation 
had  been  kept  due  in  part  to  the  fact  that  oral  presentations  had  been  used  to  streamline 
the  source  selection  process.  While  the  Comp.  Gen.  agreed  that  oral  presentations  are  an 
effective  means  of  streamlining  the  source  selection  process  and  enhancing  an  agency’s 
understanding  of  an  offeror's  approach,  the  Comp.  Gen.  voiced  the  opinion  that  the  FAR 
requirements  for  documentation  does  not  limit  the  flexibility  afforded  by  oral 
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presentations.  FAR  Sections  15.102(e)  and  15.308  do  not  require  a  particular  method  of 
establishing  a  record  of  what  was  said  by  offerors  during  oral  presentations  or  place  an 
excessive  burden  on  contracting  agencies  in  recording  oral  presentations.  Whatever 
method  is  chosen,  FAR  Sections  15.102(e),  15-305(a)  and  15-308  do  establish  an 
obligation  to  provide  a  reasonably  adequate  record  of  such  presentations  and  the 
evaluation  thereof.  Such  a  record  permits  the  courts  and  boards  to  perform  a  meaningful 
review  of  the  agency's  selection  decision.  In  this  case,  the  oral  presentations  constituted 
the  offerors'  entire  technical  proposals  (the  only  written  portions  of  proposals  related  to 
past  performance).  The  record  of  the  oral  presentations  and  the  evaluation  is  so  sketchy, 
that  the  Comp.  Gen.  had  no  means  to  determine,  based  upon  the  record,  the 
reasonableness  of  the  agency's  selection.  Therefore,  the  protest  of  the  source  selection 
decision  was  sustained  where  the  record  did  not  establish  the  reasonableness  of  the 
evaluation  or  the  cost/technical  tradeoff  underlying  the  source  selection.  The  ruling  was 
based  upon  the  principle  that: 

CGP  7a.  Agency  evaluation  judgments  must  be  documented  in  sufficient 
detail  to  allow  review  of  the  merits  of  a  protest,  to  show  that  they  are  not 
arbitrary,  and  to  show  that  they  are  in  accord  with  the  evaluation  criteria 
listed  in  the  RFP.  [Ref.  26] 

b.  Matter  of:  Support  Services,  Inc.,  B-282407;  B-2 824 07.2,  July  8, 
1999 

Support  Services,  Inc.  (protestor)  protested  the  Naval  Facilities 
Engineering  Command's  (agency)  award  of  a  contract  to  DGR  Associates,  Inc.,  (awardee) 
for  maintenance  and  repair  of  240  military  family  housing  units  at  the  Marine  Corps 
Support  Activity  in  Belton,  Missouri.  The  protestor  challenges  the  evaluation  of  past 
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performance/relevant  experience  and  argues  that  the  award  was  based  on  a  defective 
price/past  performance  tradeoff. 

During  the  review  of  the  source  selection  records,  the  Comp.  Gen.  found 
that  the  written  documentation  provided  by  the  agency  lacked  sufficient  detail  for  the 
Comp.  Gen  to  make  a  determination.  Therefore,  the  Comp.  Gen.  had  to  request  clarifying 
statements  from  the  contracting  officer.  This  move  by  the  Comp.  Gen.  was  based  upon 
earlier  protest  rulings  that  stated  that  the  Comp.  Gen.  would  accord  more  weight  to 
contemporaneous  documents  in  determining  whether  an  evaluation  was  reasonable,  post- 
protest  explanations  that  were  found  to  be  credible  and  consistent  with  the 
contemporaneous  documentation  would  generally  be  considered  in  the  review. 

In  this  case,  the  contracting  officer's  post-protest  explanations  were  found 

to  be  generally  consistent  with  the  contemporaneous  evaluation  record  and  were 

sufficient  to  support  the  conclusion  that  all  three  of  the  protester's  submitted  housing 

maintenance  references  were  considered.  It  is  undisputed  by  the  protester  that  the 

contemporaneous  evaluation  record  confirms  the  contracting  officer's  post-protest 

statement  that  she  documented  the  Portsmouth  Naval  Shipyard  reference  ratings  with 

notations  on  the  respective  reference  questionnaire  submitted  by  the  protester. 

Furthermore,  the  contemporaneous  evaluation  records  provided  no  indication  that  the 

protestor's  quote  was  rejected  for  failure  to  provide  at  least  three  references  (as  were  the 

other  quotes  that  failed  to  include  the  required  references).  The  records  indicated  that  the 

firm's  quote  was  considered  with  the  other  acceptable  quotes  in  the  best  value 

determination.  Consequently,  while  the  Comp.  Gen.  agreed  that  the  language  in  the 

contracting  officer's  source  selection  statement  suggested  that  the  Portsmouth  contract 
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was  not  considered,  since  the  record  shows  that  the  contracting  officer  was  well  aware  of 
the  Portsmouth  reference  when  she  made  her  source  selection  decision,  the  Comp.  Gen. 
concluded  that  the  protestor's  quote  was  not  rejected  for  failure  to  provide  three 
references,  and  that  the  Portsmouth  reference  was  factored  into  the  decision,  as  the 
contracting  officer  declares  in  her  statement  in  response  to  the  protest. 

Notwithstanding  the  lack  of  evidence  in  contemporaneous  written 
evaluation  records  concerning  the  number  of  references  considered,  the  record  as  a  whole 
indicated  that  all  relevant  references  furnished  by  the  protester  were  considered.  The 
agency  reasonably  evaluated  the  awardee's  experience  with  larger  housing  maintenance 
contracts  more  favorably  than  the  protester's,  and  the  agency  reasonably  determined  that 
the  protester's  experience  performing  interstate  rest  area  maintenance  work  was  not 
relevant,  since  the  contract  to  be  awarded  was  for  housing  maintenance  and  repair.  The 
Comp.  Gen.  denied  the  protest  against  the  evaluation  of  the  protester's  past 
performance/relevant  experience,  and  this  ruling  was  based  in  part  upon  the  principle 
that: 

CGP  7b.  When  contemporaneous  documentation  does  not  completely 
address  the  issues  at  hand,  the  Comptroller  General  will  consider  post¬ 
protest  explanations  from  contracting  officials  that  are  credible  and 
consistent  with  the  documentation.  [Ref.  33] 

F.  SUMMARY 

This  chapter  has  covered  the  basic  steps  in  the  protest  process.  It  has  provided  an 
analysis  of  past  performance  protests  from  Fiscal  Year  1997  through  the  first  three 
quarters  of  Fiscal  Year  2000  (Appendix  C).  It  provided  a  breakdown  of  past  performance 
protests  from  Fiscal  Year  1998  through  June  of  Fiscal  Year  2000,  and  it  concluded  by 
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categorizing  and  reviewing  these  past  performance  protests  in  order  to  identify  key  case 
principles  (Appendix  D). 

By  recognizing  that  clear  and  unambiguous  communication  of  the  evaluation 
factors  in  the  solicitation,  consistency  in  the  application  and  evaluation  of  these  factors, 
and  the  documentation  of  the  evaluations  and  the  best  value  determinations  related  to 
these  factors  are  at  the  heart  of  the  lessons  that  can  be  learned  from  these  Comp.  Gen., 
Federal  acquisition  professionals  can  more  effectively  incorporate  the  use  of  PPI  into 
their  acquisition  and  contracting  processes.  The  next  chapter  will  tie  these  case  principles 
and  the  best  practices  identified  in  Chapter  III  with  a  model  of  the  Federal  acquisition 
process  to  act  as  a  tool  in  assisting  Federal  acquisition  professionals  to  more  effectively 
utilize  PPI. 
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V.  AN  INTEGRATED  MODEL  OF  THE  FEDERAL 
ACQUISITION  PROCESS 

A.  INTRODUCTION 

The  goals  of  the  Federal  acquisition  process  are  to  satisfy  the  customer’s 
requirements  for  goods  and  services  in  a  timely  and  cost  effective  manner,  while 
minimizing  business  and  technical  risk,  accomplishing  socioeconomic  objectives, 
maximizing  competition,  and  maintaining  the  integrity  of  the  procurement  process.  [Ref. 
15:p.  5-6]  With  these  goals  in  mind,  this  chapter  introduces  the  researcher’s  conceptual 
model  of  the  Federal  acquisition  process  (Appendix  E). 

To  provide  Federal  acquisition  professionals  with  a  common  frame  of  reference, 
the  format  for  the  model  was  adopted  from  the  model  of  the  Federal  acquisition  process 
developed  by  the  Federal  Acquisition  Institute  and  presented  in  their  guidebook  The 
Federal  Acquisition  Process  (March  1999).  [Ref.  15]  The  content  of  the  model  has  been 
modified  to  reflect  the  Federal  acquisition  process  as  it  is  currently  taught  at  the  Naval 
Postgraduate  School.  Information  from  the  following  courses  was  incorporated  into  the 
model  for  the  identification  of  the  phases  and  the  tasks: 

•  MN  3303  Principles  of  Acquisition  and  Contracting 

•  MN  3304  Contract  Pricing  and  Negotiations 

•  MN  3315  Acquisition  Management  and  Contract  Administration 

•  MN  3331  Principles  of  Systems  Acquisition  and  Program  Management 

The  model  presents  the  Federal  acquisition  process  as  a  three-phased  process 
(Procurement  Planning,  Contract  Formation,  and  Contract  Administration)  that  flows 
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through  twelve  distinct  steps.  These  twelve  steps  are  in  turn  composed  of  eighty-six 
individual  functions  and  tasks  that  can  be  tailored  to  meet  the  needs  of  each  procurement. 
These  functions  and  tasks  identify  both  the  acquisition  and  the  contracting  activities  that 
must  be  completed  for  the  successful  acquisition  of  both  goods  and  services. 

In  presenting  the  model,  the  researcher  will  only  focus  on  those  functions  and 
tasks  that  either  have  an  impact  on  the  collection  and  use  of  PPI  or  are  impacted  by  the 
collection  and  use  of  PPI.  The  narrative  of  the  model  will  follow  a  set  pattern.  Each 
function  or  task  will  be  identified,  and  a  brief  description  of  the  purpose  for  the  function 
or  task  will  be  provided.  Then  the  guidance  provided  by  the  FAR  for  that  function  or 
task  will  be  presented.  Following  the  presentation  of  the  FAR  requirements,  the  best 
practices  (BP)  and/or  the  Comptroller  General  case  principles  (CGP)  that  may  apply  to 
the  function  or  task  being  reviewed  will  be  presented. 

The  researcher’s  intent  for  creating  this  model  is  that  by  integrating  those 
functions  or  tasks  related  to  the  collection  and  use  of  PPI  with  the  general  and  specific 
guidance  provided  by  the  mandatory  documents,  the  best  practices,  and  the  Comp.  Gen. 
principles  identified  in  this  thesis.  Federal  acquisition  professionals  will  be  able  to  more 
effectively  incorporate  the  use  of  PPI  into  their  acquisition  and  contracting  processes  to 
achieve  the  goals  of  the  process.  Lastly,  this  model  embraces  the  spirit  of  the  statement  of 
guiding  principles  contained  in  FAR  1.102(d): 

The  role  of  each  member  of  the  acquisition  team  is  to  exercise  personal 
initiative  and  sound  business  judgment  in  providing  the  best  value  product 
or  service  to  meet  the  customer’s  needs.  In  exercising  initiative. 
Government  members  of  the  acquisition  team  may  assume  if  a  specific 
strategy,  practice,  policy,  or  procedure  is  in  the  best  interest  of  the 
Government  and  is  not  addressed  in  the  FAR,  nor  prohibited  by  law. 
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Executive  order  or  other  regulation,  then  that  strategy,  practice,  policy,  or 
procedure  is  a  permissible  exercise  of  authority. 

B.  PROCUREMENT  PLANNING 
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Table  5.1.  Procurement  Planning.  After  Ref.  [  1 5] 

1.  Acquisition  Integrated  Product  Teams  (Items  8,  9,  &  10) 

The  purpose  of  an  acquisition  Integrated  Product  Team  (IPT)  is  to  effectively  and 
efficiently  carry  out  the  processes  necessary  for  the  procurement  of  goods  and  services. 
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DoDD  5000.1.D.l.b  directs  that  Program  Managers  (PMs)  and  other  acquisition 
managers  shall  apply  the  concept  of  Integrated  Product  and  Process  Development  (IPPD) 
throughout  the  acquisition  process  to  the  maximum  extent  practicable,  and  at  the  core  of 
IPPD  implementation  are  IPTs.  DoDD  50001. 1.D.3.C  goes  on  to  further  state  that 
Defense  acquisition  works  best  when  all  of  the  Department’s  components  work  together. 
Cooperation  and  empowerment  are  essential.  The  Department’s  acquisition  community 
shall  implement  the  concepts  of  IPPD  and  IPTs  as  extensively  as  possible. 

Acquisition  managers  should  refer  to  best  practice  number  two  (BP  2)  when 
forming  these  IPTs.  It  encourages  acquisition  managers  to  invest  in  command  or  program 
resources  needed  for  a  competent  and  well-documented  best  value  source  selection,  and  it 
reminds  them  that  they  should: 

•  Include  the  source  selection  authority  as  an  active  participant 

•  Train  all  the  evaluators  in  best  practices 

In  today’s  acquisition  environment  these  IPTs  will  in  many  cases  be  matrix 
organizations,  and  the  team  members  will  come  from  diverse  backgrounds.  Because  of 
the  rapid  pace  of  acquisition  reform,  IPT  members  will  bring  varying  levels  of  experience 
and  education  to  their  respective  teams.  It  is  critical  that  any  deficiencies  in  the  training 
of  evaluators  are  identified  up  front  and  early.  By  identifying  deficiencies  early  on  in  an 
acquisition  process,  the  assigned  contracting  official  can  provide  training  and  guidance 
on  the  use  of  the  current  best  practices. 

Acquisition  managers  also  need  to  ensure  that  the  program  has  the  resources  in 
place  for  a  competent  and  well-documented  best  value  source  selection.  Members  of  the 
various  IPTs  should  be  provided  with  the  appropriate  resources,  so  that  they  can  be 
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actively  involved  in  their  areas  of  responsibility.  Furthermore,  because  the  final  award 
determination  is  the  responsibility  of  the  Source  Selection  Authority  (SSA),  he  or  she 
should  be  an  active  participant  in  the  process.  FAR1 5.303(b)  states  that:  The  SSA  shall  - 

•  Establish  an  evaluation  team,  tailored  for  the  particular  acquisition, 
which  includes  appropriate  contracting,  legal,  logistics,  technical, 
and  other  expertise  to  ensure  a  comprehensive  evaluation  of  offers. 

•  Approve  the  source  selection  strategy  or  acquisition  plan,  if 
applicable,  before  solicitation  release. 

•  Ensure  consistency  among  the  solicitation  requirements,  notices  to 
offerors,  proposal  preparation  instructions,  evaluation  factors  and 
subfactors,  solicitation  provisions  or  contract  clauses,  and  data 
requirements. 

When  dealing  with  issues  of  past  performance,  the  analysis  of  the  PPI  protests  has 
made  it  clear  that  constancy  in  the  collection  and  evaluation  of  past  performance  is  one  of 
the  major  areas  that  the  Comp.  Gen.  will  review  when  determining  whether  a  source 
selection  determination  was  either  fair  and  reasonable  or  arbitrary  and  capricious. 

It’s  clear  from  the  guidance  provided  by  DoD  5000.2,  the  FAR,  BP  2,  and  the 
Comp.  Gen.  case  principles  that  for  acquisition  IPTs  to  be  successful  in  efficiently  and 
effectively  acquiring  goods  and  services,  they  must  be  composed  of  acquisition 
professionals  who  are  knowledgeable  in  their  areas  of  expertise,  empowered  to  make 
decisions,  engaged  in  the  acquisition  process,  and  educated  in  the  collection,  evaluation, 
and  use  of  PPI. 

2.  Non-Cost  or  Price  Related  Factors  (Item  21) 

The  purpose  for  the  use  of  evaluation  factors  is  to  represent  the  key  areas  of 
importance  and  emphasis  to  be  considered  in  the  source  selection  decision,  and  to  provide 
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a  means  of  comparison  of  benefits  among  competing  proposals.  [Ref.lO:p.L-3] 
Evaluation  criteria  should  be  tailored  to  meet  the  needs  of  each  acquisition,  and  only 
those  factors,  which  provide  a  means  of  discriminating  between  proposals,  should  be 
selected.  The  chosen  factors  should  focus  on  the  true  discriminators  between  proposals, 
and  with  regards  to  PPI,  FAR  15.304(c)  states  that: 

•  Except  as  set  forth  in  paragraph  (c)(3)(iii)  of  this  section,  past 
performance  shall  be  evaluated  in  all  source  selections  for 
negotiated  competitive  acquisitions  issued  on  or  after  January  1, 

1999,  for  acquisitions  expected  to  exceed  $100,000.  Agencies 
should  develop  phase-in  schedules  that  meet  or  exceed  this 
schedule. 

•  Past  performance  need  not  be  evaluated  if  the  contracting  officer 
documents  the  reason  past  performance  is  not  an  appropriate 
evaluation  factor  for  the  acquisition  (OFPP  Policy  Letter  92-5). 

So  unless  it  is  documented  otherwise,  past  performance  is  a  mandatory  factor  in 
competitively  negotiated  procurements.  Agencies  are  granted  a  great  degree  of  flexibility 
in  selecting  PPI  subfactors  because  the  FAR  does  not  specifically  identify  them.  To  aid 
in  the  selection  of  PPI  subfactors,  BP  1  recommends  that  agencies  should  consider  using 
the  following  subfactors  for  the  evaluation  of  past  performance  or  relative  experience: 

•  Quality  of  performance 

•  Cost  performance 

•  Schedule  performance 

•  Business  relations 

CGP  2  reaffirms  that  agencies  enjoy  broad  discretion  in  selecting  evaluation 

criteria,  and  that  the  Comp.  Gen.  will  not  object  to  a  solicitation's  evaluation  scheme  so 

long  as  it  reasonably  relates  to  the  agency’s  needs.  CGP  3a  extends  this  ruling  by  stating 

that  agencies  have  the  discretion  to  determine  the  scope  of  the  offerors’  performance 
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histories  to  be  considered,  provided  that  all  of  the  proposals  are  evaluated  on  the  same 
basis  and  consistent  with  the  solicitation  requirements. 

3.  Development  of  the  Source  Selection  Plan  (Item  30) 

The  Source  Selection  Plan  (SSP)  is  crucial  to  the  entire  source  selection  process. 
It  has  the  following  purposes: 

•  It  specifies  the  Government’s  approach  for  soliciting  and  evaluating 
proposals 

•  It  describes  the  source  selection  organizational  structure 

•  It  designates  the  persons  who  will  perform  the  evaluation 

•  After  approval  by  the  SSA,  the  SSP  is  the  “charter”  that  the  Source 
Selection  Advisory  Council  (SSAC),  the  Source  Selection  Evaluation 
Board  (SSEB),  and  the  contracting  officer  will  follow  [Ref.  10:p.  M-6] 

The  SSP  normally  consists  of  two  parts.  The  first  part  describes  the  organization 
of  the  source  selection  team.  Typically  the  source  selection  team  consists  of  the  SSA,  the 
SSAC,  and  the  SSEB.  The  SSEB  is  further  subdivided  into  groups  for  the  evaluation  of 
cost  and  cost  realism,  technical  approach,  and  past  performance.  The  functional  areas  and 
the  responsibilities  of  each  of  the  teams  is  documented  in  detail.  The  second  part  of  the 
SSP  identifies  the  evaluation  factors  and  the  detailed  procedures  for  the  evaluation  of  the 
proposals.  Previous  editions  of  the  FAR  contained  FAR  15.612,  which  provided  the 
minimum  required  elements  of  the  SSP.  However,  the  July  2000  edition  of  the  FAR  no 
longer  contains  this  section.  The  only  guidance  for  the  SSP  is  found  in  FAR  15.303(b), 
which  directs  that,  the  SSA: 

•  Establish  an  evaluation  team,  tailored  for  the  particular  acquisition, 
which  includes  appropriate  contracting,  legal,  logistics,  technical, 
and  other  expertise  to  ensure  a  comprehensive  evaluation  of  offers. 
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•  Approve  the  source  selection  strategy  or  acquisition  plan,  if 
applicable,  before  solicitation  release. 

This  limited  guidance  provided  by  the  current  FAR  allows  the  drafters  of  SSPs  a 
great  deal  of  latitude  when  developing  evaluation  plans.  As  long  as  the  SSP  meets  any 
other  Agency,  Department,  or  local  selection  guidance,  the  drafters  of  SSPs  can  be 
innovative  and  creative  when  developing  evaluation  plans.  These  evaluation  plans  can  be 
tailored  to  meet  the  needs  of  the  individual  procurements,  and  CGPs  la  &  lb  reaffirm 
that  these  SSP  evaluation  plans  are  internal  documents. 

la.  Alleged  deficiencies  in  the  application  of  an  agency’s  evaluation 
plan  does  not  provide  a  basis  for  questioning  the  validity  of  the  award 
selection;  these  plans  axe  internal  agency  instructions  and  as  such  do  not 
give  outside  parties  any  rights.  Consequently,  the  fact  that  an  agency  may 
not  have  followed  its  internal  evaluation  instructions  is  not  a  valid  basis 
for  protest. 

lb.  As  a  preliminary  matter,  it  is  the  solicitation,  not  the  internal 
evaluation  plan  that  is  the  touchstone  for  whether  offerors  have  been 
treated  fairly  in  an  evaluation.  Requirements  stated  in  evaluation  and 
source  selection  plans  are  not  disclosed  to  offerors,  and  thus,  do  not  give 
outside  parties  any  rights. 

The  SSP  is  clearly  meant  to  be  an  internal  document  and  its  purpose  is  to  guide 
the  various  members  of  the  source  select  teams.  As  an  internal  document  the  Comp.  Gen. 
has  ruled  that  a  SSP  does  not  give  unsuccessful  offers  any  legal  rights  to  challenge 
deviations  from  the  SSP. 

C.  CONTRACT  FORMATION 

1.  Pre-Solicitation  Exchanges  with  Industry  (Items  31, 32, 33,  &  34) 


The  purposes  for  conducting  pre-solicitation  exchanges  are  to  improve 
understanding  of  Government  requirements  and  industry  capabilities,  thereby  allowing 
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potential  offerors  to  judge  whether  or  not  they  can  satisfy  the  Government's  requirements, 
and  enhancing  the  Government's  ability  to  obtain  quality  supplies  and  services  at 
reasonable  prices,  and  increase  efficiency  in  proposal  preparation,  proposal  evaluation, 
negotiation,  and  contract  award. 

FAR  15.201(c)  encourages  early  exchanges  with  industry.  It  states  that: 


Agencies  are  encouraged  to  promote  early  exchanges  of  information  about 
future  acquisitions.  An  early  exchange  of  information  among  industry  and 
the  program  manager,  contracting  officer,  and  other  participants  in  the 
acquisition  process  can  identify  and  resolve  concerns  regarding  the 
acquisition  strategy,  including  proposed  contract  type,  terms  and 
conditions,  and  acquisition  planning  schedules;  the  feasibility  of  the 
requirement,  including  performance  requirements,  statements  of  work,  and 
data  requirements;  the  suitability  of  the  proposal  instructions  and 
evaluation  criteria,  including  the  approach  for  assessing  past  performance 
information;  the  availability  of  reference  documents;  and  any  other 
industry  concerns  or  questions.*** 

FAR  15.201(c)  also  provides  a  listing  of  recommended  techniques  for  promoting 
early  exchanges  with  industry: 


•  Industry  or  small  business  conferences; 

•  Public  hearings; 

•  Market  research,  as  described  in  part  1 0; 

•  One-on-one  meetings  with  potential  offerors  (any  that  are 
substantially  involved  with  potential  contract  terms  and  conditions 
should  include  the  contracting  officer;  also  see  paragraph  (f)  of  this 
section  regarding  restrictions  on  disclosure  of  information); 

•  Pre-solicitation  notices; 

•  Draft  RFPs; 

•  RFIs; 
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Pre-solicitation  or  pre-proposal  conferences;  and 


•  Site  visits. 
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T able  5.2.  Contract  F ormation.  After  Ref.  [15] 

BP  3  echoes  the  recommendations  of  FAR  15.201.  The  best  practice  guides 


overwhelmingly  recommended  conducting  pre-solicitation  exchanges  of  information  with 
industry  to  provide  the  procuring  agency  with  the  opportunity  to  explain  to  potential 


offerors  the  proposed  approach  (es)  for  the  evaluation  of  performance  risk.  The  guides 
point  out  that  pre-solicitation  exchanges  can  help  to  ensure  that  potential  offerors  have  a 
clear  understanding  of  how  past  performance  will  be  evaluated. 

While  pre-solicitation  exchanges  are  encouraged,  procurement  integrity  must  be 
maintained  throughout  the  exchanges.  No  single  offeror  should  be  granted  a  competitive 
advantage  because  of  information  received  during  the  exchanges.  The  Comp.  Gen.  has 
ruled  on  two  issues  that  could  arise  due  to  pre-solicitation  exchanges:  the  modification  of 
evaluation  factors  arising  from  pre-solicitation  exchanges,  and  bias  on  the  part  of 
Government  officials.  CGP  2  states  the  Comp.  Gen.’s  position  on  evaluation  criteria: 

Agencies  enjoy  broad  discretion  in  selecting  evaluation  criteria  and  the 
Comp.  Gen.  will  not  object  to  a  solicitation’s  evaluation  scheme  so  long  as 
it  reasonably  relates  to  the  agency’s  needs.  The  fact  that  a  solicitation's 
requirements  or  evaluation  criteria  may  favor  the  agency's  actual  needs, 
and  the  advantage  enjoyed  by  a  particular  firm  is  not  the  result  of 
improper  government  action. 

CGP.  6  states  reflects  the  high  regard  that  the  Comp.  Gen.  holds  for  the  integrity  of 
Federal  acquisition  professionals: 

Because  Government  officials  are  presumed  to  act  in  good  faith,  we  do  not 
attribute  unfair  or  prejudicial  motives  to  them  on  the  basis  of  inference  or 
supposition.  Where  a  protester  alleges  bias  on  the  part  of  Government 
officials,  the  protester  must  provide  credible  evidence  clearly 
demonstrating  a  bias  against  the  protester  or  for  the  awardee  and  showing 
that  the  agency's  bias  translated  into  action  that  unfairly  affected  the 
protester’s  competitive  position. 

Pre-solicitation  exchanges  with  industry  are  an  effective  tool,  and  as  long  as  the 
exchanges  are  open  and  the  same  information  is  provided  to  all  potential  offerors, 
Government  acquisition  professionals  should  feel  comfortable  about  conducting  pre¬ 
solicitation  exchanges. 
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2. 


Final  Solicitation  Preparation  (Item  35) 


Solicitations  are  used  to  communicate  the  Government’s  requirements  to 
prospective  offerors. 

FAR  15.404  (d)  &  (e)  contains  the  specific  guidance  on  what  factors  and  related 
information  must  be  in  the  final  solicitation  released  to  offerors.  These  two  passages 
direct  that: 

(d)  All  factors  and  significant  subfactors  that  will  affect  contract 
award  and  their  relative  importance  shall  be  stated  clearly  in  the 
solicitation.  The  rating  method  need  not  be  disclosed  in  the  solicitation. 

The  general  approach  for  evaluating  past  performance  information  shall  be 
described. 

(e)  The  solicitation  shall  also  state,  at  a  minimum,  whether  all 
evaluation  factors  other  than  cost  or  price,  when  combined,  are  — 

( 1 )  Significantly  more  important  than  cost  or  price; 

(2)  Approximately  equal  to  cost  or  price;  or 

(3)  Significantly  less  important  than  cost  or  price. 

FAR  15.305.2  (ii)  describes  the  mandatory  contents  of  the  solicitation  as  it  relates 
to  past  performance.  It  states: 

The  solicitation  shall  describe  the  approach  for  evaluating  past 
performance,  including  evaluating  offerors  with  no  relevant  performance 
history,  and  shall  provide  offerors  an  opportunity  to  identify  past  or 
current  contracts  (including  Federal,  State,  and  local  government  and 
private)  for  efforts  similar  to  the  Government  requirement.  The 
solicitation  shall  also  authorize  offerors  to  provide  information  on 
problems  encountered  on  the  identified  contracts  and  the  offeror’s 
corrective  actions.  The  Government  shall  consider  this  information,  as 
well  as  information  obtained  from  any  other  sources,  when  evaluating  the 
offeror  past  performance.  The  source  selection  authority  shall  determine 
the  relevance  of  similar  past  performance  information. 


BP  4  encourages  the  drafters  of  solicitations  to  communicate  effectively  with 
potential  offerors.  The  language  and  terms  used  should  be  straightforward  and  readily 
understood.  New  entrants  to  the  markets  should  be  encouraged  to  identify  other  related 
past  performance  information.  By  incorporating  these  two  approaches  into  the 
solicitation,  potential  misunderstandings  and  possible  protests  can  be  avoided. 

CGP  lb  illustrates  the  importance  that  the  Comp.  Gen.  places  on  solicitations: 

As  a  preliminary  matter,  it  is  the  solicitation,  not  the  internal  evaluation 
plan  that  is  the  touchstone  for  whether  offerors  have  been  treated  fairly  in 
an  evaluation. 

The  Comp.  Gen.  reaffirmed  the  importance  of  the  role  of  the  solicitation  with 
CGP  3a  and  CGP  4a,  which  state: 

CGP  3a.  Where  a  solicitation  requires  the  evaluation  of  offerors'  past 
performance,  an  agency  has  the  discretion  to  determine  the  scope  of  the 
offerors'  performance  histories  to  be  considered  provided  all  proposals  are 
evaluated  on  the  same  basis  and  consistent  with  the  solicitation 
requirements. 

CGP  4a.  When  making  “Best  Value  Determinations”,  the  Contracting 
Officer’s  decisions  concerning  tradeoffs  between  past  performance  and 
cost/price  will  be  judged  to  be  reasonable  and  consistent,  provided  that  it 
can  be  shown  that  the  tradeoffs  are  in  the  best  interest  of  the  government 
and  that  they  are  consistent  with  the  solicitation’s  evaluation  scheme. 

While  solicitations  are  used  to  communicate  the  Government’s  requirements  to 
prospective  offerors,  the  drafters  of  these  documents  must  keep  in  mind  that  the 
requirements  must  be  clearly  written  and  easily  understood.  The  drafters  need  to  ensure 
that  all  the  factors  and  subfactors  that  will  be  used  to  evaluate  proposals  are  identified, 
and  that  the  general  approach  for  evaluating  the  identified  factors  is  described.  Lastly, 

Federal  acquisition  professionals  need  to  understand  that  the  Comp.  Gen.  views  the 
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solicitation  as  the  “touchstone”  between  prospective  offerors  and  the  Federal 
Government.  The  Comp.  Gen.  will  hold  Federal  agencies  accountable  for  deviations 
from  either  the  evaluation  factors  presented  in  solicitations  or  deviations  from  the  relative 
weights  assigned  to  evaluations  in  the  solicitations. 

3.  Awarding  without  Discussions  (Clarifications)  (Item  45) 

Awarding  without  discussions  is  a  streamlined  method  for  conducting 
competitively  negotiated  acquisitions.  When  contracting  officials  utilize  this  method 
they  expect  to  receive  an  offeror’s  best  proposal  on  the  initial  submission,  thereby 
removing  the  requirement  for  negotiations.  FAR  15.306(a)  covers  clarifications  and 
awarding  without  discussions: 


•  Clarifications  are  limited  exchanges,  between  the  Government  and 
offerors,  that  may  occur  when  award  without  discussions  is 
contemplated. 

•  If  award  will  be  made  without  conducting  discussions,  offerors 
may  be  given  the  opportunity  to  clarify  certain  aspects  of  proposals 
(e.g.,  the  relevance  of  an  offeror’s  past  performance  information 
and  adverse  past  performance  information  to  which  the  offeror  has 
not  previously  had  an  opportunity  to  respond)  or  to  resolve  minor 
or  clerical  errors. 

•  Award  may  be  made  without  discussions  if  the  solicitation  states 
that  the  Government  intends  to  evaluate  proposals  and  make  award 
without  discussions.  If  the  solicitation  contains  such  a  notice  and 
the  Government  determines  it  is  necessary  to  conduct  discussions, 
the  rationale  for  doing  so  shall  be  documented  in  the  contract  file 
(see  the  provision  at  52.215-1). 

The  best  practices  guides  that  dealt  with  this  subject  repeated  the  requirements  of 
15.306(a).  The  researcher  did  not  review  any  cases  that  dealt  specifically  with  protests 
on  this  subject,  but  CGP  7a  could  reasonably  be  applied  to  protests  in  this  area.  Just  as  an 
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agency’s  judgments  must  be  documented  in  sufficient  detail  to  allow  review  of  the  merits 
of  a  protest,  so  too  must  clarifications  be  documented.  If  a  protestor  argued  that  an 
agency  conducted  misleading  exchanges  concerning  the  protestor’s  past  performance, 
then  it  is  safe  to  assume  that  in  reviewing  the  records  the  Comp.  Gen.  would  be  looking 
for  sufficient  detail  in  the  records  concerning  the  exchanges  to  either  sustain  or  deny  the 
protestor’s  claim. 

4.  Source  Selection  Evaluation  Board  Findings  (Item  47)  &  Source 
Selection  Advisory  Council’s  Comparative  Evaluation  (Item  52) 

Proposal  evaluation  is  an  ongoing  process  that  begins  with  the  receipt  of  the 
proposals  and  concludes  with  the  evaluation  of  final  proposal  revisions.  Its  purpose  is  to 
assess  the  quality  of  each  offer  and  to  determine  the  capability  of  each  offeror  [Ref.  8:p. 
820]. 


FAR  15.305  provides  evaluators  with  the  general  requirements  for  proposal 
evaluation. 

Proposal  evaluation  is  an  assessment  of  the  proposal  and  the  offeror's 
ability  to  perform  the  prospective  contract  successfully.  An  agency  shall 
evaluate  competitive  proposals  and  then  assess  their  relative  qualities 
solely  on  the  factors  and  subfactors  specified  in  the  solicitation. 
Evaluations  may  be  conducted  using  any  rating  method  or  combination  of 
methods,  including  color  or  adjectival  ratings,  numerical  weights,  and 
ordinal  rankings.  The  relative  strengths,  deficiencies,  significant 
weaknesses,  and  risks  supporting  proposal  evaluation  shall  be  documented 
in  the  contract  file. 

FAR  15.305  (a)(2)  provides  evaluators  with  specific  guidance  for  conducting  PPI 
evaluations. 


•  Past  performance  information  is  one  indicator  of  an  offeror's 
ability  to  perform  the  contract  successfully.  The  currency  and 
relevance  of  the  information,  source  of  the  information,  context  of 
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the  data,  and  general  trends  in  contractor's  performance  shall  be 
considered.  This  comparative  assessment  of  past  performance 
information  is  separate  from  the  responsibility  determination 
required  under  subpart  9. 1 . 


The  evaluation  should  take  into  account  past  performance 
information  regarding  predecessor  companies,  key  personnel  who 
have  relevant  experience,  or  subcontractors  that  will  perform  major 
or  critical  aspects  of  the  requirement  when  such  information  is 
relevant  to  the  instant  acquisition. 

In  the  case  of  an  offeror  without  a  record  of  relevant  past 
performance  or  for  whom  information  on  past  performance  is  not 
available,  the  offeror  may  not  be  evaluated  favorably  or 
unfavorably  on  past  performance. 

The  evaluation  should  include  the  past  performance  of  offerors  in 
complying  with  subcontracting  plan  goals  for  small  disadvantaged 
business  (SDB)  concerns  (see  Subpart  19.7),  monetary  targets  for 
SDB  participation  (see  19.1202),  and  notifications  submitted  under 
19.1202-4(b) 


Best  practices  5,  6,  &  7  add  further  clarification  and  guidance  on  the  subject  of 
conducting  and  documenting  the  evaluation  of  past  performance. 

BP  5  encourages  evaluators  to  use  the  most  relevant,  recent  past  performance 
information  available  in  making  the  source  selection  decision.  It  points  out  that  PPI  can 
come  from  a  variety  of  sources: 

•  Other  Federal  Agencies  and  Government  databases 

•  State  and  Local  Governments 

•  Commercial  contractors 

•  References  provided  by  the  offeror 

•  Quality  certificates  and  awards 
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BP  6  recommends  that  evaluators  conduct  reference  checks  and  look  for  patterns 
or  trends.  Whenever  possible  evaluators  are  encouraged  to  request  two  POCs  for  each 
reference,  and  they  are  further  encouraged  to  use: 

•  Questionnaires 

•  Face  to  face  interviews.  Keep  in  mind  that  these  provide  more  insight  than 
other  methods;  they  can  expose  problems  with  the  contractor  that  can  be 
discussed  during  negotiations. 

•  Telephone  interviews 

•  Some  combination  of  the  above  methods 

BP  7  strongly  recommends  that  all  evaluators  document  strengths,  weaknesses, 
risks,  and  the  associated  value  of  proposals  to  support  the  cost/past  performance  tradeoff, 
and  that  they  recognize  the  fact  that  numerical  rating  systems  can  imply  false  precision 
and  limit  flexibility  in  the  decision  making  process 

In  handling  protests  that  may  arise  over  the  evaluation  of  PPI,  the  Comp.  Gen.  can 
be  expected  to  apply  CGP  3c,  3d,  5,  7a,  &  7b  when  appropriate. 

CGP  3  a  &3b.  (a)  There  is  no  requirement  that  an  agency  contact  all  of  an 
offeror's  references,  (b)  An  agency  may  not  ignore  prior  performance 
information  of  which  it  is  aware. 

CGP  5.  The  evaluation  of  proposals  is  within  the  discretion  of  the 
procuring  agency  since  it  is  responsible  for  defining  its  needs  and  the  best 
method  of  accommodating  them,  and  must  bear  the  burden  arising  from  a 
defective  evaluation.  The  Comptroller  General’s  function  is  to  examine 
the  agency  evaluation  to  ensure  that  it  was  reasonable  and  consistent  with 
the  stated  evaluation  factors. 

CGP  7a  &  7b.  (a)  Agency  evaluation  judgments  must  be  documented  in 
sufficient  detail  to  allow  review  of  the  merits  of  a  protest,  to  show  that 
they  are  not  arbitrary,  and  to  show  that  they  are  in  accord  with  the 
evaluation  criteria  listed  in  the  RFP.  (b)  When  contemporaneous 
documentation  does  not  completely  address  the  issues  at  hand,  the 
Comptroller  General  will  consider  post-protest  explanations  from 
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contracting  officials  that  are  credible  and  consistent  with  the 

documentation. 

The  evaluation  of  an  offeror’s  PPI  is  one  of  the  most  subjective  areas  in  the 
Federal  acquisition  process.  While  the  FAR  provides  guidance  on  the  subject,  the  best 
practices  guides  provide  additional  information  on  the  methods  for  the  collection  and 
documentation  of  PPI.  The  Comp.  Gen.  principles  make  it  clear  that  consistency  in  the 
application  of  the  evaluation  factors  contained  in  the  solicitation,  and  the  documentation 
of  the  evaluation  process  are  critical  in  mitigating  the  possibility  of  sustained  protests. 

5.  Competitive  Ranges  Determinations  (Communications)  (Item  48) 

Competitive  range  determinations  are  conducted  throughout  the  evaluation 
process.  The  purposes  for  establishing  a  competitive  range  during  an  acquisition  are  to 
narrow  the  range  to  only  those  most  highly  rated  proposals,  or  to  further  reduce  the  range 
for  the  purposes  of  efficiency. 

FAR  15.306(b)  covers  communications  with  offerors  before  establishment  of  the 
competitive  range.  Communications  are  exchanges,  between  the  Government  and 
offerors,  after  receipt  of  proposals,  leading  to  establishment  of  the  competitive  range.  If  a 
competitive  range  is  to  be  established,  these  communications  - 

•  Shall  be  limited  to  the  offerors  described  in  paragraphs  (b)(l)(i) 
and  (b)(l)(ii)  of  this  section  and  -- 

•  Shall  be  held  with  offerors  whose  past  performance  information  is 
the  determining  factor  preventing  them  from  being  placed  within 
the  competitive  range.  Such  communications  shall  address  adverse 
past  performance  information  to  which  an  offeror  has  not  had  a 
prior  opportunity  to  respond. 
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The  researcher  did  not  come  across  any  best  practices  that  provided  more  specific 
guidance  than  is  already  provided  by  FAR  15.306(b).  However,  the  research  indicates 
that  the  Comp.  Gen.  can  be  expected  to  invoke  CGP  6,  7a&b,  when  dealing  with  PPI 
protests  in  this  area.  Specifically,  the  Comp.  Gen.  will  be  looking  into  the  issue  of  bias  on 
the  part  of  Government  officials  and  the  issue  of  whether  there  is  sufficient  evidence  in 
the  record  to  support  the  exclusion  of  an  offeror  from  the  competitive  range.  CGP  6 
states  that: 

Because  government  officials  are  presumed  to  act  in  good  faith,  we  do  not 
attribute  unfair  or  prejudicial  motives  to  them  on  the  basis  of  inference  or 
supposition.  Where  a  protester  alleges  bias  on  the  part  of  government 
officials,  the  protester  must  provide  credible  evidence  clearly 
demonstrating  a  bias  against  the  protester  or  for  the  awardee  and  showing 
that  the  agency’s  bias  translated  into  action  that  unfairly  affected  the 
protester's  competitive  position. 

CGP  7a  &  7b  also  remind  Government  acquisition  professionals  that: 

a.  Agency  evaluation  judgments  must  be  documented  in  sufficient 
detail  to  allow  review  of  the  merits  of  a  protest,  to  show  that  they  are  not 
arbitrary,  and  to  show  that  they  are  in  accord  with  the  evaluation  criteria 
listed  in  the  RFP. 

b.  When  contemporaneous  documentation  does  not  completely 
address  the  issues  at  hand,  the  Comptroller  General  will  consider  post¬ 
protest  explanations  from  contracting  officials  that  are  credible  and 
consistent  with  the  documentation. 

In  today’s  competitive  environment,  many  companies  may  bid  on  Government 
contracts,  and  some  of  these  companies  may  not  have  a  reasonable  chance  of  winning  the 
award  due  to  poor  PPI.  With  the  reduction  in  the  Federal  acquisition  workforce,  the 
establishment  of  a  competitive  range  allows  Government  acquisition  professionals  to 
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maximize  the  use  of  their  resources  by  focusing  on  only  those  most  highly  rated 
proposals. 


6.  Source  Selection  Authority’s  Final  Award  Determination  (Item  53) 

The  purpose  of  the  SSA’s  source  selection  final  award  determination  is  to  select 

the  offer  or  offers  which  represent  the  best  value  to  the  Government.  In  many  cases,  this 
will  be  achieved  through  a  tradeoff  process.  The  tradeoff  decision  by  the  SSA  reflects 
the  Government’s  willingness  to  accept  a  higher  priced  offer  because  the  perceived 
benefits  of  the  higher  priced  offer  are  in  the  best  interest  of  the  Government. 

FAR  15.308  guides  the  actions  of  the  SSA.  It  directs  that: 

The  source  selection  authority’s  (SSA)  decision  shall  be  based  on  a 
comparative  assessment  of  the  proposals  against  all  source  selection 
criteria  in  the  solicitation.  While  the  SSA  may  use  reports  and  analyses 
prepared  by  others,  the  source  selection  decision  shall  represent  the  SSA’s 
independent  judgment.  The  source  selection  decision  shall  be 
documented,  and  the  documentation  shall  include  the  rationale  for  any 
business  judgments  and  tradeoffs  made  or  relied  on  by  the  SSA,  including 
benefits  associated  with  additional  costs.  Although  the  rationale  for  the 
selection  decision  must  be  documented,  that  documentation  need  not 
quantify  the  tradeoffs  that  led  to  the  decision. 

BP  8  &  9  apply  to  source  selection  decisions.  They  amplify  the  requirements  from 

FAR  15.308.  BP  8  recommends  justifying  price  premiums  with  tradeoff  documentation 

regardless  of  the  selected  proposal’s  cost  or  past  performance  superiority.  It  further 

provides  a  rule  of  thumb  that  should  be  considered: 

•  The  greater  the  price  premium,  the  greater  the  documentation  must  show 
the  worth 

BP  9  strongly  recommends  that  the  SSA  ensures  that  the  source  selection  decision 
is  consistent  with  the  relative  weights  assigned  to  the  evaluation  factors  in  the 
solicitation. 
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CGP  4  a,  b,  c,  6,  and  7a&b  need  to  be  taken  into  consideration  when  the  SSA 
makes  the  final  award  determination.  CGP  4  a,  b,  c  deal  with  best  value  determinations; 
they  state  that: 

a.  When  making  “Best  Value  Determinations”,  the  Contracting 
Officer’s  decisions  concerning  tradeoffs  between  past  performance  and 
cost/price  will  be  judged  to  be  reasonable  and  consistent,  provided  that  it 
can  be  shown  that  the  tradeoffs  are  in  the  best  interest  of  the  government 
and  that  they  are  consistent  with  the  solicitation’s  evaluation  scheme. 

b.  When  price  is  secondary  to  non-cost/price  factors,  an  agency  may 
not  select  a  lower-priced,  lower-scored  proposal,  unless  it  determines  that 
the  premium  involved  in  selecting  a  higher-rated,  higher-priced  proposal  is 
not  justified.  However,  when  selection  officials  reasonably  regard 
proposals  as  being  essentially  equal,  price  can  become  the  determining 
factor  in  making  award  notwithstanding  that  the  evaluation  criteria 
assigned  price  less  importance  than  non-cost/price  factors. 

c.  In  a  negotiated  procurement  with  a  best  value  evaluation  plan, 
point  scores  are  useful  as  guides,  but  they  do  not  mandate  automatic 
selection  of  a  particular  proposal.  Whether  a  given  point  spread  between 
two  competing  proposals  indicates  a  significant  superiority  of  one 
proposal  over  another  depends  upon  the  facts  and  circumstances  of  each 
procurement  and  is  primarily  a  matter  within  the  discretion  of  the 
procuring  agency.  The  question  of  whether  a  difference  in  point  scores  is 
significant  is  for  determination  on  the  basis  of  what  that  difference  might 
mean  in  terms  of  contract  performance  and  what  it  would  cost  the 
government  to  take  advantage  of  it. 

CGP  6  deals  with  the  issue  bias  on  the  part  of  Government  officials,  and  it  states 
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Because  Government  officials  are  presumed  to  act  in  good  faith,  we  do  not 
attribute  unfair  or  prejudicial  motives  to  them  on  the  basis  of  inference  or 
supposition.  Where  a  protester  alleges  bias  on  the  part  of  government 
officials,  the  protester  must  provide  credible  evidence  clearly 
demonstrating  a  bias  against  the  protester  or  for  the  awardee  and  showing 
that  the  agency's  bias  translated  into  action  that  unfairly  affected  the 
protester's  competitive  position. 
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CGP  7a  &  7b  deal  with  the  issues  that  may  arise  over  the  documentation  of  the 
final  award  determination.  They  highlight  the  fact  that: 

a.  Agency  evaluation  judgments  must  be  documented  in  sufficient 
detail  to  allow  review  of  the  merits  of  a  protest,  to  show  that  they  are  not 
arbitrary,  and  to  show  that  they  are  in  accord  with  the  evaluation  criteria 
listed  in  the  RFP. 

b.  When  contemporaneous  documentation  does  not  completely 
address  the  issues  at  hand,  the  Comptroller  General  will  consider  post¬ 
protest  explanations  from  contracting  officials  that  are  credible  and 
consistent  with  the  documentation. 

The  SSA’s  final  award  determination  is  one  of  the  last  tasks  before  a  contract  is 
awarded.  In  acquisitions  involving  best  value  determinations,  the  documentation  of 
tradeoffs  that  led  to  the  determination  are  critical.  While  the  Comp.  Gen.  has  stated  that 
Government  officials  are  presumed  to  act  in  good  faith,  the  merits  of  a  protest  will  in 
large  part  be  determined  by  the  content  of  the  acquisition’s  documentation. 

7.  Debriefings  (Item  58) 

The  purpose  for  debriefing  unsuccessful  offerors  is  to  inform  them  of  the  reasons 
why  their  offers  were  unsuccessful,  and  to  help  them  identify  areas  in  their  proposals 
where  improvements  could  be  made  for  future  acquisitions. 

Once  a  final  award  determination  has  been  made,  the  time  for  being  innovative 
and  creative  has  passed.  FAR  15.506  provides  very  explicit  direction  for  the  post  award 
debriefing  of  Offerors.  It  covers  such  topics  as:  the  time  restrictions  for  when  debriefs 
must  take  place,  the  methods  contracting  officers  may  use  for  conducting  debriefs,  the 
minimum  requirements  for  what  information  may  be  debriefed,  and  detailed  restrictions 
on  what  information  may  not  be  disclosed. 


82 


With  regards  to  debriefs,  BP  10  only  adds  sage  advice  on  the  reasons  for 
conducting  proper  and  timely  debriefs: 

•  Debriefings  provide  unsuccessful  offerors  with  the  opportunity  to  learn 
about  their  strengths  and  weaknesses  and  how  to  improve  future  proposals 
submitted  to  the  Government 

•  Offerors  rely  heavily  on  information  provided  during  debriefing  sessions 
to  influence  their  decisions  regarding  filing  protests 

Debriefings  are  a  necessary  function  in  the  acquisition  process.  Not  every 
company  that  submits  a  proposal  has  experience  dealing  within  the  Federal  acquisition 
process,  and  for  some  it  is  a  learning  experience.  As  the  Comp.  Gen.  has  pointed  out, 
“The  function  of  debriefing  is  not  to  justify  or  defend  a  selection  decision,  but  to  assist 
offerors  in  preparing  future  proposals.”  [Ref.  10:p.  H-l  1] 

8.  Protests  (Item  59) 

The  requirements  for  the  current  protest  procedures  have  their  roots  in  the 
Competition  in  Contracting  Act  of  1984  (CICA).  CICA’s  bid  protest  procedures  enable 
disappointed  bidders  to  compel  Federal  acquisition  professionals  to  explain  their 
procurement  decisions  to  the  Comp.  Gen. 

As  with  protests,  the  time  for  innovation  and  creativity  are  over.  The  FAR 
requirements  are  very  lock-stepped.  FAR  33.104  provides  very  specific  guidance  that 
must  be  followed  by  Federal  acquisition  professionals  in  the  event  of  a  protest  to  GAO 
Comp.  Gen.  It  provides  general  procedures,  and  then  it  provides  very  detailed  and 
specific  procedures  for  both  protests  that  arise  prior  to  the  awarding  of  a  contract  and 
protests  that  arise  after  the  awarding  of  a  contract. 
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A  comprehensively  documented  acquisition  contract  file  is  the  key  to  mitigating 
the  risk  of  a  sustainable  protest.  BP  7  &  BP  8  stress  the  need  for  documenting  the 
strengths,  weaknesses,  and  risks  associated  with  each  proposal.  They  also  stress  the  need 
for  documenting  the  justifications  for  any  price  premiums  that  may  result  from  tradeoffs 
between  cost  or  price  and  past  performance.  CGP  7a  drives  home  the  point  that  when  the 
documentation  does  not  completely  address  the  issues  at  hand,  the  Comp.  Gen.  will  only 
consider  post-protest  explanations  from  Federal  acquisition  professionals  that  are  credible 
and  consistent  with  the  documentation. 

D.  CONTRACT  ADMINISTRATION 

1.  Documenting  Contractor  Performance  &  Closeout  (Items  70  &  85) 

The  purpose  for  documenting  a  contractor’s  performance  information  periodically 
during  contract  performance  is  to  motivate  the  contractor  to  maintain  high  quality 
performance  or  to  improve  inadequate  performance  before  the  next  reporting  period. 
[Ref.5:  p.  2]  The  purpose  for  documenting  a  contractor’s  performance  information  at  the 
closeout  of  a  contract  is  to  provide  agencies  with  a  final  assessment  of  a  contractor’s 
overall  performance.  Both  the  interim  and  final  performance  information  can  be  used  as 
source  selection  information  by  Federal  agencies. 

The  policies  for  the  documentation  of  past  performance  information  are  found  in 
FAR  42.15.  The  goal  of  these  policies  is  to  ensure  a  clear  and  concise  recording  of  a 
contractor’s  performance  on  every  contract,  task  order,  or  other  contractual  document 
exceeding  $100,000,  based  on  a  discussion  with  the  contractor  about  recent  performance. 
[Ref.  5:p.  4] 
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The  specific  procedures  that  govern  the  collection  of  PPI  are  found  in  FAR 


42.1503  they  state  that: 


•  Agency  procedures  for  the  past  performance  evaluation  system 
shall  generally  provide  for  input  to  the  evaluations  from  the 
technical  office,  contracting  office  and,  where  appropriate,  end 
users  of  the  product  or  service. 

•  Agency  evaluations  of  contractor  performance  prepared  under  this 
subpart  shall  be  provided  to  the  contractor  as  soon  as  practicable 
after  completion  of  the  evaluation.  Contractors  shall  be  given  a 
minimum  of  30  days  to  submit  comments,  rebutting  statements,  or 
additional  information.  Agencies  shall  provide  for  review  at  a 
level  above  the  contracting  officer  to  consider  disagreements 
between  the  parties  regarding  the  evaluation.  The  ultimate 
conclusion  on  the  performance  evaluation  is  a  decision  of  the 
contracting  agency.  Copies  of  the  evaluation,  contractor  response, 
and  review  comments,  if  any,  shall  be  retained  as  part  of  the 
evaluation.  These  evaluations  may  be  used  to  support  future  award 
decisions,  and  should  therefore  be  marked  "Source  Selection 
Information".  The  completed  evaluation  shall  not  be  released  to 
other  than  Government  personnel  and  the  contractor  whose 
performance  is  being  evaluated  during  the  period  the  information 
may  be  used  to  provide  source  selection  information. 

•  Departments  and  agencies  shall  share  past  performance 
information  with  other  departments  and  agencies  when  requested 
to  support  future  award  decisions.  The  information  may  be 
provided  through  interview  and/or  by  sending  the  evaluation  and 
comment  documents  to  the  requesting  source  selection  official. 

•  Any  past  performance  information  systems,  including  automated 
systems,  used  for  maintaining  contractor  performance  information 
and/or  evaluations  should  include  appropriate  management  and 
technical  controls  to  ensure  that  only  authorized  personnel  have 
access  to  the  data. 

•  The  past  performance  information  shall  not  be  retained  to  provide 
source  selection  information  for  longer  than  three  years  after 
completion  of  contract  performance. 
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I  CONTRACT  ADMINISTRATION  | 

Initiation  of  Work 
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Contract 
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Property 
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61.  Post-Award 

67.  Delays 

72.  Assignment  of 

8 1 .  Intellectual 

84.  Terminations 

Orientation 

Claims 

Property 

Conferences 

62.  Consent  to  Sub- 

68.  Stop  Work 

73.  Limitation  of 

85.  Closeout 

contracts 

Costs 

Other 

69.  Commercial  & 

74.  Unallowable 

82.  Administering 

86.  Fraud  & 

Modifications 

Noncommercial 

Costs 

Socio-Economic 

Exclusion 

Remedies 

Requirements 

63.  Contract 

70.  Documenting 

75.  Profit  and  Fee 

Modifications 

Contractor 

Adjustments 

64.  Options 

Performance 

76.  Collecting 
Contractor  Debts 

65.  Task  and 
Delivery  Order 

Accounting 

Contracts 

77.  Accounting 
and  Estimating 

fflggn 

Accounting 

Standards 

79.  Defective 
Pricing 

Table  5.3.  Contract  Administration.  After  Ref.  [  1 5] 

BP  11  stresses  the  point  that  assessing  current  contract  performance  is  a  basic 


“best  practice”  for  good  contract  administration,  and  it  is  one  of  the  most  important  tools 
available  for  ensuring  good  contractor  performance. 

CGP  3b  provides  further  clarification  for  the  limitations  placed  on  the  retention  of 
PPI  stated  in  FAR  42.1502.  The  Comp.  Gen.  has  ruled  that  a  contractor’s  past 
performance  may  be  considered  for  up  to  three  years  after  the  completion  of  contract 
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performance  as  a  whole,  rather  than  for  only  three  years  after  each  incident  of 
performance  under  the  contract. 

With  the  increased  emphasis  on  the  use  of  PPI  in  making  source  selection 
determinations,  the  documentation  of  contractor  performance  will  become  an 
increasingly  vital  function  in  the  Federal  acquisition  process.  Federal  acquisition 
professionals  need  to  have  a  clear  understanding  not  only  of  the  FAR  requirements,  but 
also  how  those  requirements  have  been  modified  by  Comp.  Gen.  rulings. 

E.  SUMMARY 

This  chapter  introduced  the  researcher’s  conceptual  model  of  the  Federal 
acquisition  model.  The  three  major  phases  and  the  twelve  steps  of  the  process  were 
identified,  and  then  eighty-six  functions  and  tasks  associated  with  the  steps  were 
identified.  The  researcher  integrated  the  best  practices  and  the  Comp.  Gen.  principles 
into  those  functions  and  tasks  that  either  have  an  impact  on  the  collection  and  use  of  PPI 
or  are  impacted  by  the  collection  and  use  of  PPI. 

Once  again,  the  researcher’s  intent  for  developing  this  integrated  model  was  to 
present  Federal  acquisition  professionals  with  a  tool  that  effectively  integrated  PPI  best 
practices  and  Comp.  Gen.  principles  into  their  acquisition  and  contracting  activities. 
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VI.  SUMMARY  AND  RECOMMENDATIONS 


A.  INTRODUCTION 

This  chapter  provides  a  summary  of  the  research  presented  in  this  thesis  by 
reviewing  the  primary  and  the  secondary  research  questions.  Each  question  is  restated, 
and  then  the  answer(s)  that  were  developed  during  the  research  are  presented.  The 
chapter  concludes  with  the  researcher’s  recommended  areas  for  further  study  and 
analysis. 

B.  SUMMARY 

The  primary  research  question  of  this  thesis  dealt  with  the  question  of,  “What 
would  an  analysis  of  published  best  practices  guides  and  Comptroller  General  case 
decisions  suggest  are  the  best  overall  practices  and  key  case  principles  for  effective  past 
performance  information  management,  and  how  could  these  best  practices  and  case 
principles  be  integrated  into  the  acquisition  and  contracting  processes  of  a  buying 
activity?” 

Five  best  practices  and  best  value  guides  were  reviewed  in  Chapter  III,  and  from 
these  guides,  eleven  overall  best  practices  were  identified.  These  overall  best  practices 
are  listed  in  Appendix  “A”.  In  Chapter  IV,  one  hundred  and  fifty-one  protests  that  dealt 
with  issues  of  past  performance  were  examined.  After  identifying  and  analyzing  several 
recent  trends,  the  protests  were  arranged  into  eight  categories  that  mirror  the  flow  of  tasks 
in  the  Federal  acquisition  process.  From  these  eight  categories,  fourteen  cases  were 
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selected  as  examples  to  highlight  the  prevailing  case  principles  that  the  Comp.  Gen.  has 
relied  upon  in  determining  the  validity  of  protests  that  involve  issues  of  PPI. 

Chapter  V  introduced  the  researcher’s  interpretation  of  the  current  model  of  the 
Federal  acquisition  process.  The  overall  best  practices  identified  in  Chapter  III  and  the 
Comp.  Gen.  case  principles  from  Chapter  IV  were  integrated  into  the  appropriate  phases 
and  tasks  of  the  Federal  acquisition  process  to  illustrate  how  Federal  acquisition 
professionals  can  more  effectively  utilize  PPI  in  their  acquisition  and  contracting 
processes. 

The  first  secondary  research  question  raised  the  question  of,  “How  have  policies 
evolved  with  respect  to  past  performance?” 

The  research  presented  in  Chapter  II  of  this  thesis  has  shown  that  in  the  eight 
years  since  the  release  of  OFPP  Policy  Letter  92-5,  the  requirements  for  the  collection 
and  use  of  PPI  have  become  more  stringent.  While  in  1995  the  threshold  for  the 
collection  and  use  of  PPI  was  established  for  contracts  at  or  above  $1,000,000,  that 
threshold  has  now  been  lowered  to  contracts  that  are  at  or  above  $100,000. 

The  second  secondary  research  question  asked,  “What  best  practice  guides  are 
available  that  provide  guidance  on  the  collection  and  evaluation  of  past  performance 
information?” 

In  Chapter  III,  it  was  shown  that  there  are  numerous  best  practices  guides 
available  to  the  Federal  acquisition  workforce.  Because  of  the  rapid  pace  of  acquisition 
reform  and  the  sometimes-myopic  focus  of  the  drafters  of  these  guides,  Federal 
acquisition  professionals  are  encouraged  to  review  them  to  ensure  that  they  are  current 
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and  accurate  before  using  them.  Also  because  no  single  Service  or  Agency  has  a 
comprehensive  lessons  learned  database,  acquisition  professionals  should  be  encouraged 
to  review  several  of  the  guides  to  gain  a  fuller  understanding  of  the  overall  best  practices 
for  the  collection  and  use  of  PPI. 

The  third  secondary  research  question  posed  the  question,  “What  are  the 
recommended  methods  for  the  evaluation  of  past  performance?” 

During  the  review  of  the  best  practices  guides  in  Chapter  III,  it  was  noted  time 
and  again  that  at  the  heart  of  the  evaluation  process  is  the  verification  and  validation  of 
each  offeror’s  PPI.  Past  performance  evaluators  are  encouraged  to  use  the  most  current 
and  relevant  PPI  available.  PPI  can  be  obtained  from  a  variety  of  sources: 

•  Other  Federal  Agencies  and  Government  databases 

•  State  and  Local  Governments 

•  Commercial  contractors 

•  References  provided  by  the  offeror 

•  Quality  certificates  and  awards 

PPI  evaluators  are  instructed  to  conduct  reference  checks  and  to  look  for  patterns 
or  trends.  Whenever  possible  they  are  encouraged  to  contact  two  points  of  contact  (POC) 
for  each  reference.  Time  permitting,  face-to-face  interviews  are  the  preferred  method  for 
conducting  reference  checks,  because  they  can  provide  more  insight  into  the  offeror’s 
past  performance  than  other  methods. 

Because  the  evaluation  of  PPI  is  highly  subjective  in  nature,  the  best  practices 
guides,  reviewed  by  the  researcher,  overwhelmingly  agreed  that  adjectival  ratings 
schemes  with  narratives  are  the  preferred  method  for  recording  the  evaluation  of  PPI. 


91 


Several  of  the  guides  recognized  that  numerical  rating  systems  can  falsely  imply 
precision  and  limit  the  flexibility  of  the  decision  making  process.  All  the  guides  stressed 
that  no  matter  which  method  of  evaluation  was  used,  all  evaluations  should  be  clearly 
documented  to  provide  the  Source  Selection  Authority  with  a  clear  picture  of  the 
strengths,  weaknesses,  and  risks  associated  with  each  offeror’s  proposal. 

The  fourth  secondary  research  question  looked  at,  “How  does  industry  view  the 
use  of  PPI  by  Federal  acquisition  professionals?” 

The  literature  review  and  the  survey  of  industry  representatives  presented  in 
Chapter  III,  and  the  interview  with  Deputy  Under  Secretary  of  Defense  (Acquisition 
Reform)  presented  in  Chapter  IV  all  indicate  that  businesses  are  suspicious  of  the  motives 
behind  the  use  of  PPI  by  Federal  procurement  personnel.  One  of  their  biggest  concerns 
centers  on  the  issue  of  being  treated  fairly  in  the  selection  process.  A  large  majority  of  the 
businesses  that  deal  with  the  Federal  Government  feel  that  the  FAR  allows  too  much 
latitude  to  evaluators,  resulting  in  widely  differing  weighting  and  selection  of  evaluation 
factors  among  Federal  agencies  and  military  commands.  [Ref.  9:p.  31]  Businesses  want  a 
level  playing  field  with  regards  to  PPI,  and  they  feel  that  they  cannot  be  guaranteed  one 
with  the  current  PPI  policies  and  the  current  PPI  collection  and  tracking  systems. 

The  fifth  secondary  research  question  dealt  with  the  question  of,  “Under  what 
circumstances  is  an  offeror  likely  to  file  a  protest  over  the  use  of  PPI  by  a  Federal 
acquisition  professional?” 

The  research  presented  in  Chapter  V  indicates  that  in  today’s  competitive 
environment  a  company’s  bid  and  proposal  money  is  a  precious  commodity,  and  when 
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companies  decide  to  bid  on  Government  contracts,  they  are  making  a  conscious  business 
decision  to  win  the  contract.  When  PPI  is  weighted  heavily  as  a  source  selection  factor, 
the  research  in  this  thesis  indicates  that  companies  may  increase  the  level  of  risk  that  they 
are  willing  to  accept  by  adjusting  their  pricing  strategies  in  order  to  win  the  contract. 

With  this  added  risk  and  the  pressures  to  acquire  contracts  in  competitive  markets, 
companies  are  more  likely  to  file  protests  over  the  collection  and  use  of  PPI,  whenever 
there  is  the  appearance  that  the  integrity  of  the  procurement  process  has  been  breached, 
(e.g.,  whenever  there  is  the  perception  that  the  evaluators  failed  to  evaluate  all  of  the 
proposals  on  the  same  basis  and  consistent  with  the  solicitation  requirements). 

Also  in  highly  competitive  markets  where  there  are  businesses  hungry  for  work, 
companies  are  more  likely  to  file  protests  if  the  best  value  determinations  to  either  pay  or 
not  to  pay  price  premiums  appear  to  be  inconsistent  with  the  weights  assigned  to  the 
source  selection  factors  identified  in  the  solicitations.  Furthermore,  in  the  absence  of 
timely  and  productive  debriefs,  unsuccessful  offerors  may  initially  file  protests  to 
determine  if  the  PPI  evaluations  and  the  tradeoffs  that  led  to  the  best  value  determinations 
had  been  documented  in  the  contract  file. 

The  sixth  secondary  research  question  asked,  “What  are  the  discemable  trends  in 
the  number  of  sustained  protests  involving  past  performance  in  recent  years?” 

Chapter  IV  dealt  with  PPI  protests  that  had  been  brought  before  the  Comp.  Gen. 
from  Fiscal  Year  1997  through  June  2000.  This  timeframe  was  chosen  because  it 
coincided  with  the  shifts  in  policy  that  began  to  place  great  emphasis  on  the  collection 
and  use  of  PPI.  The  research  presented  in  Table  4.1  indicates  that  the  number  of  past 
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performance  protests  as  a  percentage  of  merit  protests  has  increased  from  8%  in  1997  to 
30%  in  June  2000.  The  research  also  indicates  that  the  number  of  past  performance 
protests  as  a  percentage  of  the  total  sustained  protests  has  also  seen  a  corresponding 
increase  from  10%  in  Fiscal  Year  1997  to  30%  in  Fiscal  Year  2000. 

The  seventh  secondary  research  question  dealt  with,  “What  elements  of  the 
acquisition  contracting  process  will  the  Comp.  Gen.  review  in  making  his  decisions?” 

The  research  presented  in  Chapter  IV  clearly  shows  that  the  Comp.  Gen.  will  be 
looking  for  consistency  of  action  by  acquisition  professionals  throughout  the  Federal 
acquisition  process,  and  the  Comp.  Gen.  will  be  looking  for  fair  and  impartial  treatment 
for  all  businesses  that  compete  for  Government  contracts.  When  issues  arise  over  the  PPI 
evaluation  criteria  selected  for  a  specific  procurement,  the  Comp.  Gen.  will  review  the 
source  selection  files  to  determine  if  the  evaluation  factors  were  consistent  with  the 
requirement,  and  the  Comp.  Gen.  will  review  the  solicitation  and  the  source  selection 
evaluator’s  notes  to  ensure  that  the  evaluation  criteria  were  consistently  applied  to  each 
offeror.  The  Comp.  Gen.  has  stated  that  the  evaluation  of  proposals  is  within  the 
discretion  of  the  procuring  agency  since  it  is  responsible  for  defining  its  needs  and  the 
best  method  of  accommodating  them,  and  it  must  bear  the  burden  arising  from  a 
defective  evaluation.  When  protests  arise  over  the  use  of  best  value  tradeoffs,  the  Comp. 
Gen.  will  review  the  Source  Selection  Authority’s  final  determination  to  ensure  that  any 
tradeoffs  made  between  cost  and  past  performance  were  in  the  best  interest  of  the 
Government  and  that  they  were  consistent  with  the  solicitation’s  evaluation  scheme. 
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The  analysis  of  the  case  histories  also  shows  that  the  Comp.  Gen.  has  a  preference 
for  contemporaneous  written  documentation.  The  Comp.  Gen.  has  ruled  in  past  cases  that 
an  agency’s  evaluation  judgments  must  be  documented  in  sufficient  detail  to  allow 
review  of  the  merits  of  the  protest,  to  show  that  they  are  not  arbitrary,  and  to  show  that 
they  are  in  accordance  with  the  evaluation  criteria  listed  in  the  solicitation.  In  those  cases 
where  the  documentation  did  not  completely  address  the  issues  at  hand,  the  Comp.  Gen. 
has  been  willing  to  consider  post-protest  explanations  from  credible  contracting  officials 
so  long  as  the  explanations  were  consistent  with  the  documentation. 

The  eighth  secondary  research  question  asked,  “What  case  principles  can  be 
extracted  from  the  Comp.  Gen.  decisions  that  would  aid  acquisition  professionals 
throughout  the  acquisition  process?” 

In  preparing  Chapter  IV,  one  hundred  and  fifty-one  PPI  protests  were  analyzed 
and  from  this  analysis  it  was  noted  that  the  Comp.  Gen.  based  the  rulings  in  the  cases  on 
one  or  more  of  fourteen  key  case  principles.  These  principles  are  listed  in  Appendix  D. 

C.  RECOMMENDATIONS  FOR  AREAS  FOR  FURTHER  RESEARCH 

•  Conduct  a  joint  review  with  representatives  from  the  Marine  Corps 
Systems  Command  (MARCORSYSCOM)  of  the  current  reform  initiatives 
and  the  current  best  procurement  practices,  with  the  intent  of  rewriting  the 
Marine  Corps’  Best  Value  Handbook.  A  rewrite  of  the  existing  handbook 
would  provide  MARCORSYSCOM  acquisition  personnel  with  guidance 
that  is  more  reliable,  responsive,  and  relevant  to  the  current  acquisition 
environment. 

•  Conduct  follow-on  analysis  of  protests  brought  before  the  GAO  Comp. 
Gen.  for  fiscal  years  2001  and  beyond,  to  determine  if  there  are  any  new 
case  principles.  New  case  principles  have  the  potential  to  modify  the 
current  interpretations  of  the  statutory  (FAR)  requirements  that  govern  the 
actions  of  acquisition  professionals.  New  interpretations  of  the  statutory 
requirements  by  the  Comp.  Gen.  will  either  allow  acquisition 
professionals  more  or  less  discretion  in  carrying  out  the  tasks  required  to 
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conduct  fair  and  reasonable  procurements.  A  particular  area  to  investigate 
is  the  use  of  neutral  past  performance  evaluations  in  best  value 
procurements  where  past  performance  is  the  discriminating  evaluation 
factor.  Currently  an  offeror  without  relevant  PPI  may  not  be  evaluated 
favorably  or  unfavorably  on  the  factor  of  past  contract  performance,  but 
this  raises  the  question,  “Doesn’t  a  best  value  determination  based  on  past 
performance  as  a  key  discriminator  penalize  an  offeror  with  no  prior  past 
performance?”  This  is  an  issue  that  the  Comp.  Gen.  will  need  to  rule  on  in 
the  near  future. 

Conduct  an  analysis  of  cases  involving  past  performance  issues  brought 
before  the  Court  of  Federal  Claims,  to  identify  case  principles  and  trends. 
Here  again  the  intent  would  be  to  analyze  the  cases  to  determine  if  there 
are  any  key  case  principles  that  would  assist  Federal  acquisition 
professionals  to  more  effectively  incorporate  the  collection  and  use  of  PPI 
into  the  acquisition  process.  Trends  that  are  identified  here  would  also 
indicate  how  well  Federal  acquisition  professionals  are  doing  with  regards 
to  the  collection  and  use  of  PPI. 

Conduct  an  analysis  of  past  performance  protests  that  have  been  resolved 
through  the  use  of  alternative  disputes  resolution  (ADR)  methods,  to  gain 
insight  into  the  total  population  of  past  performance  protests.  An  analysis 
of  the  total  population  of  PPI  protests  would  provide  greater  insight  into 
how  well  the  Federal  acquisition  workforce  has  institutionalized  the  use  of 
PPI  in  source  selections. 

Conduct  a  cost  analysis  of  past  performance  protests  to  determine  the  true 
costs  of  these  protests  in  terms  of  lost  procurement  time  and  money.  If  one 
of  the  goals  of  the  Revolution  in  Business  Affairs  is  to  deliver  high  quality 
goods  and  services  in  less  time  and  at  a  lower  total  cost,  then  how  has  the 
increased  reliance  on  PPI  assisted  the  Federal  acquisition  community  in 
achieving  that  goal?  Follow  on  research  could  look  at  what  are  the 
potential  costs  and  benefits  of  either  increasing  or  decreasing  the 
requirements  for  the  collection  and  use  of  PPI. 

Conduct  an  analysis  of  future  Federal  Legislative  actions  or  Executive 
Branch  reform  initiatives  that  may  potentially  impact  the  collection  and 
use  of  past  performance  information  by  the  Federal  acquisition  workforce. 
This  area  of  research  goes  hand  in  hand  with  the  previous  area.  Here  the 
researcher  would  be  analyzing  the  proposed  legislation  or  reform 
initiatives  to  determine  the  associated  costs  and  benefits  that  would  have 
impact  upon  the  Federal  acquisition  community. 
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APPENDIX  A.  BEST  PRACTICE  PRINCIPLES 


BP.  1  Tailor  the  use  and  evaluation  of  past  performance  information  to  fit  the  needs  of 
the  acquisition. 

•  Weight  past  performance  to  ensure  that  it  is  a  valid  discriminator. 

•  Limit  sub-factors  to  true  discriminators.  Recommended  PPI  sub-factors 
are: 

•  Quality  of  performance 

•  Cost  performance 

•  Schedule  performance 

•  Business  relations 

BP.  2  Invest  in  command  or  program  resources  needed  for  a  competent  and  well 
documented  best  value  source  selection. 

•  Include  the  source  selection  authority  as  an  active  participant 

•  Train  all  the  evaluators  in  best  practices 

BP.  3  Conduct  pre-solicitation  exchanges  of  information  with  industry  to  explain  the 
approach  you  will  use  to  evaluate  performance  risk,  (e.g.,  draft  solicitations  & 
pre-solicitation  conferences) 

•  Pre-solicitation  exchanges  can  help  to  ensure  that  potential  offerors  have  a 
clear  understanding  of  how  past  performance  will  be  evaluated 

BP.  4  Structure  the  solicitation  to  communicate  effectively  to  potential  offerors. 

•  Encourage  new  entrants  to  the  market  to  identify  other  related  past 
performance  information 

BP.  5  Use  the  most  relevant,  recent  past  performance  information  available  in  making 
the  source  selection  decision.  PPI  can  come  from  a  variety  of  sources: 

•  Other  Federal  Agencies  and  Government  databases 

•  State  and  Local  Governments 

•  Commercial  contractors 

®  References  provided  by  the  offeror 

•  Quality  certificates  and  awards 

BP.  6  Conduct  references  checks  and  look  for  patterns  or  trends.  Whenever  possible 
request  two  POC’s  for  each  reference.  Use: 
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•  Questionnaires 

•  Face  to  face  interviews.  Keep  in  mind  that  these  provide  more  insight  than 
other  methods;  they  can  expose  problems  with  the  contractor  that  can  be 
discussed  during  negotiations 

•  Telephone  interviews 

•  Some  combination  of  the  above  methods 

BP.  7  Document  strengths,  weaknesses,  risks  and  the  associated  value  of  proposals  to 
support  the  cost/past  performance  tradeoff. 

•  Recognize  that  numerical  rating  systems  can  imply  false  precision  and 
limit  flexibility  in  the  decision  making  process 

BP.  8  Justify  price  premiums  with  tradeoff  documentation  regardless  of  the  selected 
proposal’s  cost  or  past  performance  superiority. 

•  The  greater  the  price  premium,  the  greater  the  documentation  must  show 
the  worth 

BP.  9  Ensure  that  the  source  selection  decision  is  consistent  with  the  relative  weights 
assigned  to  the  evaluation  factors  in  the  solicitation. 

BP.  10  Conduct  a  proper  and  timely  debriefing. 

•  Debriefings  provide  unsuccessful  offerors  with  the  opportunity  to  learn 
about  their  strengths  and  weaknesses  and  how  to  improve  future  proposals 
submitted  to  the  Government 

•  Offerors  rely  heavily  on  information  provided  during  debriefing  sessions 
to  influence  their  decisions  regarding  filing  protests 

BP.  1 1  Assess  current  contract  performance;  it  is  a  basic  “best  practice”  for  good  contract 
administration,  and  it  is  one  of  the  most  important  tools  available  for  ensuring 
good  contractor  performance. 
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APPENDIX  B.  PAST  PERFORMANCE  RESEARCH  SURVEY 


How  might  the  following  scenarios  affect  your  pricing  strategies? 

•  You  receive  an  RFP,  which  indicates  that  past  performance  will  be 
weighted  significantly  more  than  cost  or  price.  (Past  performance  50%, 
cost  or  price  20%,  and  technical  30%) 

•  In  the  first  case,  you  are  confident  that  your  past  performance 
rating  will  be  higher  than  your  competitors 

•  In  the  second  case,  you  feel  that  your  past  performance  rating  will 
be  somewhat  less  than  your  competitors 

•  You  receive  an  RFP,  which  indicates  that  past  performance  will  be 
weighted  significantly  less  than  cost  or  price.  (Cost  or  price  70%, 
technical  20%,  past  performance  10%) 

•  In  the  first  case,  you  are  confident  that  your  past  performance 
rating  will  be  higher  than  your  competitors 

•  In  the  second  case,  you  are  confident  that  your  past  performance  rating 
will  be  somewhat  less  than  your  competitors 
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APPENDIX  C.  PAST  PERFORMANCE  PROTESTS 


CASES  WITH  SUSTAINED  PAST  PERFORMANCE  PROTESTS 

Protestor 

B-Number 

Date 

Biospherics,  Inc. 

B-278278 

Jan.  14, 1998 

SCIENTECH,  Inc. 

B-277805.2 

Jan.  20, 1998 

Foundation  health  Federal  Services,  Inc. 

B278189.3 

Feb.  4, 1998 

U.S.  Property  Management  Service 

B-278727 

Mar.  6, 1998 

Aerospace  Design  &  Fabrication,  Inc. 

B-278896.2 

May  4,  1998 

Century  Environmental  Hygiene,  Inc. 

B-279378 

Jun.  5, 1998 

Consolidated  Engineering  Services,  Inc. 

B-279565.2 

Jun.  26, 1998 

Trifax  Corporation 

B-279561 

Jun.  29,  1998 

Technical  Support  Services,  Inc. 

B-279665 

Jul.8, 1998 

Possehn  Consulting 

B-278579.2 

Jul.  29, 1998 

Professional  Gunsmithing  Inc. 

B-279048.2 

Aug.  24, 1998 

GTS  Duratek,  Inc. 

B-28051 1.2 

Oct.  19, 1998 

Intellectual  Properties,  Inc. 

B-280803 

Nov.  19,  1998 

Teltara  Inc. 

B-280922 

Dec.  4,  1998 

MCR  Federal,  Inc. 

B-280969 

Dec.  14,  1998 

Opti-Lite  Optical 

B-281693 

Mar.  22,  1999 

ENMAX  Corporation 

B-281965 

May  12, 1999 

ACS  Government  Solutions  Group,  Inc. 

B-282098 

Jun.  2, 1999 

Universal  Building  Maintenance,  Inc. 

B-282456 

Jul.  15,  1999 

ATA  Defense  Industries,  Inc. 

B-2825 1 1 

Jul.  21,  1999 

Nova  Group,  Inc. 

B-282947 

Sep.  15,  1999 

Beneco  Enterprises,  Inc., 

B-283154 

Oct.  13, 1999 

OneSource  Energy  Services,  Inc. 

B-283445 

Nov.  19,  1999 

Kathpal  Technologies,  Inc. 

B-283137.3 

Dec.  30,  1999 

Saco  Defense  Corporation 

B-283885 

Jan.  20, 2000 

North  America,  Inc. 

B-283743.2 

Feb.  16,  2000 

CRAssociates,  Inc. 

B-282075 

Mar.  15,  2000 

Future-Tec  Management  Systems,  Inc. 

B-283793 

Mar.  20,  2000 

Maritime  Berthing,  Inc. 

B-284123.3 

Apr.  27, 2000 

Lloyd  H.  Kessler,  Inc. 

B-284693 

May  24, 2000 

Computer  Products,  Inc. 

B-284702 

May  24, 2000 

J&J  Maintenance,  Inc. 

B-284708.2 

Jun.  5, 2000 
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CASES  WITH  DENIED  PAST  PERFORMANCE  PROTESTS 


Protestor 

Dawco  Construction,  Inc 
J  &  E  Associates,  Inc 
BFI  Waste  Systems 

Compania  De  Asesoria  Y  Comerico  S.A. 
High  Country  Contracting 
U.S.  Technology  Corporation 
Davies  Rail  and  Mechanical  Works,  Inc. 
Battelle  Memorial  Institute 
Modem  Technologies  Corp. 

Xeno  Technix,  Inc 
Rockhill  Industries,  Inc. 

RAMCOR  services  Group,  Inc. 

San  Diego  Beverage  &  Kup,  Inc. 

Techsys  Corporation 
NAHB  Research  Center,  Inc. 

Aerospace  Design  &  Fabrication,  Inc. 
PCT  Services,  Inc. 

Protection  Total  /  Magnum  Security,  S.A. 
Ideal  Electronic  Security  Company 
Quaker  Valley  Meats,  Inc. 

Advanced  Data  Concepts,  Inc. 

Dual,  Inc 

WECO  Cleaning  Specialists,  Inc. 

Source  of  One  Management 
TEAM  Support  Services,  Inc. 

Hard  Bodies,  Inc. 

Hago-Cantu  Joint  Venture 
Acepex  Management  Corporation 
Pacific  Ship  Repair  and  Fabrication,  Inc. 
SDV  Telecommunications 
Apache  enterprises,  Inc. 

YKK  (U.S.A.),  Inc. 

General  Security  Services  Corporation 
Ervin  &  Associates,  Inc. 

Rohmann  Services,  Inc. 

Durocher  Dock  &  Dredge 


B-Number  Date 


B-278048.2 

B-278187 

B-278223 

B-278358 

B-278649 

B-278584 

B-278260 

B-278673 

B-278695 

B-278738 

B-278797 

B-276633.2 

B-278881 

B-278904 

B-278876 

B-278896.2 

B-279168 

B-278 129.4 

B-279221 

B-279217 

B-277801.4 

B-279295 

B-279305 

B-278044.4 

B-279379.2 

B-279543 

B-279637.2 

B-279 173.5 

B-279793 

B-279919 

B-278855.2 

B-280447 

B-280388 

B-279 16 1.2 

B-280154.2 

B-280853 


Jan.  2, 1998 
Jan.  5, 1998 
Jan.  8, 1998 
Jan.  20, 1998 
Jan.  30, 1998 
Feb.  17, 1998 
Feb.  25, 1998 
Feb.  27, 1998 
Mar.  4, 1998 
Mar.  11, 1998 
Mar.  16, 1998 
Mar.  23,  1998 
Mar.  24,  1998 
Apr.  13, 1998 
May  4, 1 998 
May  4, 1998 
May  12,  1998 
May  12,  1998 
May  19,  1998 
May  20,  1998 
Jun.  1, 1998 
Jun.  1,  1998 
Jun.  3, 1998 
Jun.  12, 1998 
Jun.  22, 1998 
Jun.  23,  1998 
Jul.  20, 1998 
Jul.  22, 1998 
Jul.  23, 1998 
Jul.  29, 1998 
Jul.  30, 1998 
Aug.  28, 1998 
Sep.  25, 1998 
Oct.  13, 1998 
Nov.  16, 1998 
Nov.  24,  1998 


102 


Protestor 

Lear  Siegler  Services,  Inc. 

Environmental  Tectonics  Corporation 
Westinghouse  Government 
Venture  Productions 
MCR  Federal.  Inc. 

PharmChem  Laboratories,  Inc 
PCT  Services,  Inc. 

RTF/TCI/EAI  Joint  Venture 
Saratoga  Medical  Center,  Inc. 

Bulova  Technologies  LLC 
Saratoga  Medical  Center,  Inc. 

Orbital  Technologies  Corporation 
Stratus  Systems,  Inc. 

Borders  Consulting,  Inc. 

Kellie  W.  Tipton  Construction  Company 
LB&L  Associates,  Inc. 

SGT.  Inc. 

Korrect  Optical 

Construction  Technology  Laboratories,  Inc. 
Marathon  Watch  Company  Limited 
Marquette  Medical  Systems,  Inc. 

Walsh  Dohmen  Southeast 
NAPA  Supply  of  Grand  Forks,  Inc. 

GCI  Information  services,  Inc 
TRW,  Inc. 

Advanced  Data  Concepts,  Inc. 

Life  Oxygen  &  Health  Services,  Inc. 
AmClyde  Engineering  Products  Co.  Inc. 
Inland  Service  Corporation 
Support  Services,  Inc. 

U.S.  Constructors,  Inc. 

AdvanChip  Corporation 
Allied  Technology  Group,  Inc. 

Olympus  Building  Services,  Inc. 

Infrared  Technologies  Corporation 
D.F.  Zee's  Fire  Fighter  Catering 
Engineering  Air  Systems 
Acepex  Management  Corporation 


B-Number 

B-280834 

B-280573.2 

B-280928 

B-280872 

B-280969 

B-280988 

B-281046 

B-280422.3 

B-281350 

B-281384 

B-281388 

B-281453 

B-281645 

B-281606 

B-281331.3 

B-281706 

B-281773 

B-281800 

B-281836 

B-281876 

B-277827.5 

B-281904 

B-280996.2 

B-282074 

B-282162 

B-280967.8 

B-282243 

B-282271 

B-282772 

B-282407 

B-282776 

B-282571 

B-282739 

B-282887 

B-282912 

B-280767.4 

B-283011 

B-283080 


Date 

Nov.  25,  1998 
Dec.  1, 1998 
Dec.  4, 1998 
Dec.  9, 1998 
Dec.  14, 1998 
Dec.  17,  1998 
Dec.  22, 1998 
Dec.  29, 1998 
Jan.  27, 1999 
Feb.  3, 1999 
Feb.  3, 1999 
Feb.  17, 1999 
Feb.  24, 1999 
Mar.  10, 1999 
Mar.  22, 1999 
Mar.  24,  1999 
Apr.  1,  1999 
Apr.  9,  1999 
Apr.  12, 1999 
Apr.  22, 1999 
Apr.  22, 1999 
Apr.  29,  1999 
May  13, 1999 
May  28, 1999 
Jun.  9, 1999 
Jun.  14, 1999 
Jun.  18, 1999 
Jun.  21,  1999 
Jun.  21, 1999 
Jul.  8,  1999 
Jul.  21, 1999 
Jul.  29, 1999 
Aug.  19,  1999 
Aug.  31, 1999 
Sep.  2, 1999 
Sep.  10, 1999 
Sep.  21, 1999 
Oct.  4, 1999 
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Protestor 

The  Communities  Group 
GSA,  Inc. 

J.A.  Jones  Grupo  de  Servicios,  SA 
J&E  Associates,  Inc. 

Rotech  Medical  Corporation 

Ideal  Electronic  Security  Company,  Inc. 

Olin  Corporation 

Crescent  Helicopters 

Simborg  Development,  Inc. 

BioGenesis  Pacific,  Inc. 

Advanced  Communications  Systems,  Inc. 
B.  Diaz  Sanitation,  Inc 
Mid  Pacific  Environmental 
Valenzuela  Engineering,  Inc. 

MG  Industries 

Eastco  Building  Services,  Inc. 

Universal  Fabric  Structures,  Inc. 

A.G.  Cullen  Construction,  Inc. 

Champion  Service  Corporation 
NV  Services 
Clean  Venture,  Inc. 

Dellew  Corporation 

Dynacs  Engineering  Company,  Inc. 

Schaeffer  Eye  Center 

Butt  Construction  Company,  Inc. 

PEMCO  World  Air  Services 
TiHu,  Inc. 

M-Cubed  Information  Systems,  Inc. 
Crown  Support  Services,  Inc. 

WinStar  Federal  Services 
Marathon  Construction  Corporation 
Molina  Engineering,  Ltd. 

KELO,  Inc. 

Information  Network  Systems,  Inc. 

Gray  Personnel  Services 


B-Number 

Date 

B-283147 

Oct.  12, 1999 

B-283177 

Oct.  18, 1999 

B-283234 

Oct.  25, 1999 

B-283448 

Nov.  3,  1999 

B-283295.2 

Nov.  8, 1999 

B-283398 

Nov.  10,  1999 

B-283401 

Nov.  15,  1999 

B-283469.2 

Nov.  30,  1999 

B-283538 

Dec.  7, 1999 

B-283738 

Dec.  14, 1999 

B-283650 

Dec.  16,  1999 

B-283 827/28  Dec.  27, 1999 

B-283309.2 

Jan.  10, 2000 

B-283 889 

Jan.  13, 2000 

B-283010.3 

Jan.  24,  2000 

B-283 972.2 

Feb.  10, 2000 

B-284032 

Feb.  10,  2000 

B-284049.2 

Feb.  22,  2000 

B-2841 16 

Feb.  22,  2000 

B-284 119.2 

Feb.  25,  2000 

B-2841 76 

Mar.  6, 2000 

B-284227 

Mar.  13,  2000 

B-284234 

Mar.  17,  2000 

B-284268 

Mar.  20,  2000 

B-284270 

Mar.  20,  2000 

B-284240.3 

Mar.  27,  2000 

B-284360 

Mar.  31,  2000 

B-284445 

Apr.  19,  2000 

B-284471 

Apr.  21,  2000 

B-284617 

May  17,  2000 

B-2848 1 6 

May  22, 2000 

B-284895 

May  22,  2000 

B-284601 

Jun.  7, 2000 

B-284854 

Jun.  12, 2000 

B-285002 

Jun.  26, 2000 
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APPENDIX  D.  COMPTROLLER  GENERAL’S  PRINCIPLES 


1.  EVALUATION  PLANS 

a.  Alleged  deficiencies  in  the  application  of  an  agency’s  evaluation  plan  does 
not  provide  a  basis  for  questioning  the  validity  of  the  award  selection; 
these  plans  are  internal  agency  instructions  and  as  such  do  not  give  outside 
parties  any  rights.  Consequently,  the  fact  that  an  agency  may  not  have 
followed  its  internal  evaluation  instructions  is  not  a  valid  basis  for  protest. 
Matter  of:  General  Security  Services  Corporation,  B-280388;  B-280388.2, 
September  25,  1998. 

b.  As  a  preliminary  matter,  it  is  the  solicitation,  not  the  internal  evaluation 
plan  that  is  the  touchstone  for  whether  offerors  have  been  treated  fairly  in 
an  evaluation.  Requirements  stated  in  evaluation  and  source  selection 
plans  are  not  disclosed  to  offerors,  and  thus,  do  not  give  outside  parties 
any  rights.  Matter  of:  Enmax  Corporation,  B-281965,  May  12, 1999. 

2.  EVALUATION  CRITERIA/FACT ORS 

Agencies  enjoy  broad  discretion  in  selecting  evaluation  criteria  and  the 
Comptroller  General  will  not  object  to  a  solicitation's  evaluation  scheme 
so  long  as  it  reasonably  relates  to  the  agency's  needs.  The  fact  that  a 
solicitation's  requirements  or  evaluation  criteria  may  favor  the  agency's 
actual  needs,  and  the  advantage  enjoyed  by  a  particular  firm  is  not  the 
result  of  improper  government  action.  Matter  of:  Borders  Consulting,  Inc., 
B-281606,  March  10,  1999. 

3.  PAST  PERFORMANCE 

a.  Where  a  solicitation  requires  the  evaluation  of  offerors'  past  performance, 
an  agency  has  the  discretion  to  determine  the  scope  of  the  offerors' 
performance  histories  to  be  considered  provided  all  proposals  are 
evaluated  on  the  same  basis  and  consistent  with  the  solicitation 
requirements.  Matter  of:  OMV  Medical,  Inc.;  Saratoga  Medical  Center, 
Inc.,  B-281606,  February  3,  1999. 

b.  A  contractor’s  past  performance  may  be  considered  for  up  to  three  years 
after  the  completion  of  contract  performance  as  a  whole,  rather  than  for 
only  three  years  after  each  incident  of  performance  under  the  contract. 
Matter  of:  D.F,  Fire  Fighter  Catering.,  B-280767.4,  September  10, 1999. 

c.  There  is  no  requirement  that  an  agency  contact  all  of  an  offeror's 
references.  Matter  of:  Kellie  W.  Tipton  Construction  Company,  B- 
281331.3,  March  22, 1999. 
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d.  An  agency  may  not  ignore  prior  performance  information  of  which  it  is 
aware.  Matter  of:  Consolidated  Engineering  Services  Inc..  B-279565.2;  B- 
279565.3,  June  26,  1998. 

4.  BEST  VALUE  EVALUATIONS 

a.  When  making  “Best  Value  Determinations”,  the  Contracting  Officer’s 
decisions  concerning  tradeoffs  between  past  performance  and  cost/price 
will  be  judged  to  be  reasonable  and  consistent,  provided  that  it  can  be 
shown  that  the  tradeoffs  are  in  the  best  interest  of  the  government  and  that 
they  are  consistent  with  the  solicitation’s  evaluation  scheme.  Matter  of: 
Marathon  Watch  Company  Limited.  B-281876;  B-28 1876.2,  April  22 
1999. 

b.  When  price  is  secondary  to  non-cost/price  factors,  an  agency  may  not 
select  a  lower-priced,  lower-scored  proposal,  unless  it  determines  that  the 
premium  involved  in  selecting  a  higher-rated,  higher-priced  proposal  is 
not  justified.  However,  when  selection  officials  reasonably  regard 
proposals  as  being  essentially  equal,  price  can  become  the  determining 
factor  in  making  award  notwithstanding  that  the  evaluation  criteria 
assigned  price  less  importance  than  non-cost/price  factors.  Matter  of: 
Beneco  Enterprises,  Inc.,  B-283154,  October  13,  1999. 

c.  In  a  negotiated  procurement  with  a  best  value  evaluation  plan,  point  scores 
are  useful  as  guides,  but  they  do  not  mandate  automatic  selection  of  a 
particular  proposal.  Whether  a  given  point  spread  between  two  competing 
proposals  indicates  a  significant  superiority  of  one  proposal  over  another 
depends  upon  the  facts  and  circumstances  of  each  procurement  and  is 
primarily  a  matter  within  the  discretion  of  the  procuring  agency.  The 
question  of  whether  a  difference  in  point  scores  is  significant  is  for 
determination  on  the  basis  of  what  that  difference  might  mean  in  terms  of 
contract  performance  and  what  it  would  cost  the  government  to  take 
advantage  of  it.  Matter  of:  Schaeffer  Eve  Center.  B-284268,  March  20 
2000. 

5.  EVALUATION  OF  OFFERS 


The  evaluation  of  proposals  is  within  the  discretion  of  the  procuring 
agency  since  it  is  responsible  for  defining  its  needs  and  the  best  method  of 
accommodating  them,  and  must  bear  the  burden  arising  from  a  defective 
evaluation.  The  Comptroller  General’s  function  is  to  examine  the  agency 
evaluation  to  ensure  that  it  was  reasonable  and  consistent  with  the  stated 
evaluation  factors.  Matter  of:  LB&B  Associates.  B-28 1706,  March  24 
1999. 
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6. 


BIAS  OF  GOVERNMENT  OFFICIALS 


Because  government  officials  are  presumed  to  act  in  good  faith,  we  do  not 
attribute  unfair  or  prejudicial  motives  to  them  on  the  basis  of  inference  or 
supposition.  Where  a  protester  alleges  bias  on  the  part  of  government 
officials,  the  protester  must  provide  credible  evidence  clearly 
demonstrating  a  bias  against  the  protester  or  for  the  awardee  and  showing 
that  the  agency's  bias  translated  into  action  that  unfairly  affected  the 
protester's  competitive  position.  Matter  of:  J.A.  Jones  Grupo  de  Servicio, 
SA,  B-283234,  October  25, 1 999. 

7.  PROPOSAL  EVALUATION-DOCUMENTATION 

a.  Agency  evaluation  judgments  must  be  documented  in  sufficient  detail  to 
allow  review  of  the  merits  of  a  protest,  to  show  that  they  are  not  arbitrary, 
and  to  show  that  they  are  in  accord  with  the  evaluation  criteria  listed  in  the 
RFP.  Matter  of:  J&J  Maintenance,  Inc.,  B-284708.2;  B-284708.3,  June  5, 
2000. 

b.  When  contemporaneous  documentation  does  not  completely  address  the 
issues  at  hand,  the  Comptroller  General  will  consider  post-protest 
explanations  from  contracting  officials  that  are  credible  and  consistent 
with  the  documentation.  Matter  of:  Support  Services,  Inc.,  B-282407;  B- 
282407.2,  July  8, 1999. 
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APPENDIX  E.  FEDERAL  ACQUISITION  PROCESS  MODEL 


PROCUREMENT  PLANNING 

Determination  of  Need 

Requirement  Definition 

Acquisition  Planning 

1.  Forecasting 

4.  Requirement 

Requirements 

Documents  (SOW/SOO) 

Acquisition  IPTs 

Selection  Factors 

2.  Purchase  Requests 

5.  Use  of  Government 

8.  Source  Selection 

20.  Cost/Price  Related 

Property  &  Supply 
Sources 

Authority 

Factors 

3.  Funding 

6.  Services 

9.  Source  Selection 

21.  Non-Cost/Price 

Advisory  Council 

Evaluation  Factors 

7.  Procurement  Request 

10.  Source  Selection 
Evaluation  Board 

Terms  and  Conditions 

1 1 .  Risk  Management 

22.  Determination  & 
Finding  (D&Fs) 

12.  Method  of 

23.  Justification  & 

Procurement 

Extent  of  Competition 

Approval  (J&As) 

24.  Contract  Types-  Pricing 

Arrangements 

13.  Competition 

25.  Recurring 

Requirements 

Requirements 

14.  Lease  vs.  Purchase 

26.  Letter  Contracts- 
Unpriced  Contracts 

15.  Commerciality 
Determination 

27.  Contract  Financing 

16.  SBA  Set-Asides 

28.  Need  for  Bonds 

17.  8(a)  Procurements 

29.  Method  of  Payment 

18.  Unsolicited  Proposals 

30.  Development  of  Source 
Selection  Plan  (SSP) 

19.  Market  Research 

30.  Development  of  Source 
Selection  Plan  (SSP) 

Modified  By  Researcher  from  [FAP,  March  1999,  pp.  5-9] 
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CONTRACT  FORMATION 

Solicitation 

Evaluation  -Sealed  Bidding 

Evaluation  -Competitive 
Proposal 

Contract  Award 

Pre-Solicitation 

Bid  Evaluation 

Proposal  Evaluation 

Selection  for  Award 

31.  Publicizing  Proposec 
Procurement 

38.  Processing  Bids 

43.  Processing  Proposals 

53.  Source  Selection 
Authority’s  Final  Award 
Determination 

32.  Draft  Solicitation 

39.  Bid  Acceptance  Periods 

44.  Responsiveness  & 

54.  Responsibility 

Preparation 

Responsibility 

Determinations 

Determination 

33.  Pre-award  Inquiries 

40.  Late  Bids 

45.  Awarding  without 
Discussions  (Clarifications) 

55.  Preparing  Award 

34.  Pre-Solicitation 
Conferences 

41.  Bid  Opening/  Bid  Prices 

46.  DCAA/  DCMA 
information 

56.  Award  and  Notices 

Solicitation  of  Offerors 

42.  Responsiveness  & 
Responsibility  Determinations 

47.  Source  Selection 
Evaluation  Board  Findings 
*Past  Performance 
*Cost 
*Technical 

57.  Mistakes  in  Offers 

35.  Final  Solicitation 

48.  Competitive  Range 

58.  Conduct  Debriefings 

Preparation 

Determination 

(Communications) 

36.  Issuance  of 
Solicitation 

49.  Fact  Finding 

59.  Protests 

37.  Amending/ 

Canceling 

Solicitation 

i 

i 

t 

( 

50.  Negotiation  Strategy 

5 1 .  Conducting 

Negotiations  (Discussions) 

52.  Source  Selection 
\dvisory  Council's 
Comparative  Evaluation 

Modified  By  Researcher  from  [FAP,  March  1999,  pp.  5-9] 
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CONTRACT  ADMINISTRATION 


Initiation  of  Work 
and  Modifications 

Quality  Assurance 

Payment  & 
Accounting 

Special  Terms 

Contract  Closeout 
or  Termination 

Planning 

Monitoring  and 
Problem  Solving 

Payment 

Property 

Administration 

Closeout 

60.  Contract 

Administration 

Planning 

66.  Monitoring, 
Inspecting,  and 
Accepting 

71.  Invoices 

80.  Government 
Furnished  Property 

83.  Claims 

61.  Post- A  ward 

Orientation 

Conferences 

67.  Delays 

72.  Assignment  of 
Claims 

81.  Intellectual 
Property 

84.  Terminations 

62.  Consent  to  Sub¬ 
contracts 

68.  Stop  Work 

73.  Limitation  of 
Costs 

Other 

85.  Closeout 

Modifications 

69.  Commercial  & 

Noncommercial 

kemedies 

74.  Unallowable  costs 

82.  Administering 

Socio-Economic 

requirements 

86.  Fraud  & 
Exclusion 

63.  Contract 
Modifications 

70.  Documenting 
Contractor  Performance 

75.  Profit  and  Fee 
Adjustments 

64.  Options 

76.  Collecting 
Contractor  Debts 

65.  Task  and 
Delivery  Order 
Contracts 

Accounting 

77.  Accounting  and 
Estimating  Systems 

78.  Cost  Accounting 
Standards 

79.  Defective  Pricing 

Modified  By  Researcher  from  [FAP,  March  1 999,  pp.  5-9] 


111 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


112 


LIST  OF  REFERENCES 


1.  A  Guide  to  Best  Practices  for  Past  Performance,  Office  of  Federal  Procurement 
Policy,  May  1995. 

2.  AMC  Pamphlet  715-3,  Contracting  for  Best  Value:  A  Best  Practices  Guide  to 
Source  Selection,  Army  Material  Command,  January  1998. 

3.  ASN  (RDA)  Memorandum,  Considerations  in  the  Use  of  Past  Performance 
Information  in  Source  Selection:  March  13, 1998. 

4.  Beausoleil,  Joseph.  “Past  Performance  is  Not  Working”.  Contract  Management 
(July  2000):  28-31. 

5.  Best  Practices  for  Collecting  and  Using  Current  and  Past  Performance,  Office  of 
Federal  Procurement  Policy,  May  2000. 

6.  Best  Practices:  DoD  Training  Can  Do  More  to  Help  Weapon  System  Programs 
Implement  Best  Practices,  United  States  Government  Accounting  Office  Report, 
August  1999. 

7.  Chew,  James.  “Commercial  Best  Practices  and  the  DOD  Acquisition  Process.” 
Acquisition  Review  Quarterly  4-2  (Spring  1997):  217. 

8.  Cibinic,  John  &  Nash,  Ralph.  Formation  of  Government  Contracts,  3rd  ed.  The 
George  Washington  University,  1998. 

9.  Clipsham,  Neil.  “Appraising  Performance  Appraisal  Systems.”  The  Military 
Engineer  591  (April-May  1998):  31-32. 

10.  Defense  Acquisition  University,  Intermediate  Contracting  (CON  202)  Student 
Guide,  2000. 

1 1 .  Defense  Acquisition  University,  Acquisition  Production  and  Quality  Management 
(PQM  201),  April  1997. 

1 2.  Department  of  Defense  Directive  5000. 1 ,  March  1 996. 

13.  Department  of  Defense  Guide  to  Collection  and  Use  of  Past  Performance 
Information:  May  1999. 

14.  Department  of  Defense  Regulation  5000.2-R,  March  1996. 

15.  “Federal  Acquisition  Process”,  Federal  Acquisition  Institute,  Government 
Printing  Office,  March  1999. 

16.  Federal  Acquisition  Regulation  (FAR),  Government  Printing  Office,  July  26, 

2000. 

17.  General  Accounting  Office,  Administrative  Practice  &  Procedure,  Bid  Protest 
Regulations,  4  CFR  Part  21,  August  08, 1996. 


113 


18.  Interview:  Mr.  Stan  Soloway,  Deputy  Under  Secretary  of  Defense  (Acquisition 
Reform),  Office  of  the  Secretary  of  Defense  (Acquisition,  Technology,  and 
Logistics),  August  31, 2000. 

19.  Matter  of:  Beneco  Enterprises,  Inc.,  B-283154,  October  13, 1999. 

20.  Matter  of:  Borders  Consulting,  Inc.,  B-28 1 606,  March  1 0,  1 999. 

21  •  Matter  of:  Consolidated  Engineering  Services  Inc.,  B-279565.2;  B-279565.3,  June 

26, 1998. 

22.  Matter  of:  D.F.  Zee’s  Fire  Fighting  Catering,  B-280767,  September  10,  1999. 

23.  Matter  of:  Enmax  Corporation,  B-28 1965,  May  12, 1999. 

24.  Matter  of:  General  Security  Services  Corporation,  B-280388;  B-280388.2, 
September  25, 1998. 

25.  Matter  of:  J.A.  Jones  Grupo  de  Servicio.  SA,  B-283234,  October  25, 1999. 

26.  Matter  of:  J&J  Maintenance,  Inc..  B-284708.2;  B-284708.3,  June  5, 2000. 

27.  Matter  of:  Kellie  W.  Tipton  Construction  Company.  B-28 1331.3,  March  22, 1 999. 

28.  Matter  of:  LB&B  Associates,  B-28 1706,  March  24,  1999. 

29.  Matter  of:  Marathon  Watch  Company  Limited.  B-28 1876:  B-28 1876.2.  April  22 
1999. 

30.  Matter  of:  OMV  Medical  Inc.:  Saratoga  Medical  Center,  Inc.,  B-28 1606, 
February  3,  1999. 

3 1 .  Matter  of:  Schaeffer  Eye  Center,  B-284268,  March  20,  2000. 

32.  Matter  of  Support  Services,  Inc.,  B-282407;  B-282407.2,  July  8,  1999. 

33.  Matter  of:  Systems  Integration  &  Research.  Inc..  B-279759;  B-279759.3, 
February  16,  1999. 

34.  Office  of  Federal  Procurement  Policy  (OFPP)  Policy  Letter  92-5:  December 
1992. 

35.  Phone  Interview:  Mr.  Jerold  D  Cohen,  Special  Assistant  to  the  General  Counsel, 
April  25,  2000. 

36.  Priced-Based  Acquisition  Report,  Office  Secretary  of  Defense,  December  1 999. 

37.  Solomon,  John  and  Pfleger,  Katherine,  “Firms  that  Defrauded  the  U.S.  Still 
Getting  Government  Contracts.”  Philadelphia  Inquirer,  24  July  2000. 

38.  Source  Selection  Activities  Guide  (Performance  Risk  Assessment  Group 
(PRAG)),  Air  Force  Material  Command,  April  2000. 

39.  Survey  Response:  Mr.  John  Callan,  Lockheed  Martin  Space  Systems  Co, 
February  16,  2000. 

40.  Survey  Response:  Mr.  Tom  Dorst,  Northrop  Grumman,  February  04, 2000. 


114 


41 .  Survey  Response:  Rick  Hawker,  ARGO  Systems,  January  3 1 , 2000. 

42.  Survey  Response:  Connie  Scott,  United  Tech  UTC-CD,  February  02, 2000. 

43.  Survey  Response:  George  West,  UDLP,  February  10, 2000. 

44.  The  Road  Ahead,  Under  Secretary  of  Defense  for  Acquisition  and  Technology, 

June  02, 2000. 

45.  U.S.  Marine  Corps  Best  Value  Source  Selection  Handbook,  Marine  Corps 
Systems  Command,  June  1997. 

46.  Web  Site:  Marine  Corps  Installations  and  Logistics  Department,  Contracts 
Division,  http://www.hqmc.usmc.mil/ilweb.nsf 

47.  Web  Site:  Navy  Business  Practices:  Past  Performance. 
http://www.abm.rda.hq.navy.mil.bpgpp.html 

48.  Web  Site:  Department  of  the  Navy  (DoN)  Contractor  Performance  Reporting 
System  (CPARS).  http://www.nslcptsmh.navsea.navy.mil/cparsmenu.htm 

49.  Web  Site:  General  Accounting  Office  (GAO),  http://www.gao.gov 

50.  Web  Site:  Where  in  Federal  Contracting,  http://www.wifcon.com 


115 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


116 


INITIAL  DISTRIBUTION  LIST 


1.  Defense  Technical  Information  Center . 2 

8725  John  J.  Kingman  Road,  Suite  0944 

Ft.  Belvoir,  VA  22060-6218 

2.  Dudley  Knox  Library . 2 


Naval  Postgraduate  School 
41 1  Dyer  Road 
Monterey,  CA  93943-5101 

3.  Director,  Training  and  Education . 1 

MCCDC,  Code  C46 

1019  Elliot  Road 
Quantico,  VA  22134-5107 

4.  Director,  Marine  Corps  Research  Center . 2 

MCCDC,  Code  C40RC 

2040  Broadway  Street 
Quantico,  VA  22134-5107 

5.  Marine  Corps  Representative . 1 

Naval  Postgraduate  School 

Code  037,  Bldg.  330,  Ingersoll  Hall,  Room  1 16 
555  Dyer  Road 
Monterey,  CA  93943 

6.  Marine  Corps  Tactical  Systems  Support  Activity . 1 

Technical  Advisory  Branch 

Attn:  Librarian 
Box  555171 

Camp  Pendleton,  CA  92055-5080 

7.  Dr.  David  V.  Lamm  (Code  SM/LT)  . 5 

Department  of  Systems  Management 

Naval  Postgraduate  School 
555  Dyer  Road 
Monterey,  CA  93943-5104 


117 


8.  Dr.  Keith  F.  Snider  (Code  SM/SK) . 2 

Department  of  Systems  Management 

Naval  Postgraduate  School 
555  Dyer  Road 
Monterey,  CA  93943-5104 

9.  Major  Mark  F.  Walkner . 2 

354  Ardennes  Circle 

Seaside,  CA  93955 

1 0.  Major  Steven  Colcombe . 1 

1 6044  Fairway  Dr. 

Dumfries,  VA  22026 


118 


